Evolving and Ensembling Deep CNN Architectures for Image Classification

Fielding, Ben, Lawrence, Tom and Zhang, Li (2019) Evolving and Ensembling Deep CNN Architectures for Image Classification. In: IJCNN 2019 - 2019 International Joint Conference on Neural Networks, 14th - 19th July 2019, Budapest, Hungary.

Fielding et al - Evolving and Ensembling Deep CNN Architectures AAM.pdf - Accepted Version

Download (1MB) | Preview


Deep Convolutional Neural Networks (CNNs) have traditionally been hand-designed owing to the complexity of their construction and the computational requirements of their training. Recently however, there has been an increase in research interest towards automatically designing deep CNNs for specific tasks. Ensembling has been shown to effectively increase the performance of deep CNNs, although usually with a duplication of work and therefore a large increase in computational resources required. In this paper we present a method for automatically designing and ensembling deep CNN models with a central weight repository to avoid work duplication. The models are trained and optimised together using particle swarm optimisation (PSO), with architecture convergence encouraged. At the conclusion of the joint optimisation and training process a base model nomination method is used to determine the best candidates for the ensemble. Two base model nomination methods are proposed, one using the local best particle positions from the PSO process, and one using the contents of the central weight repository. Once the base model pool has been created, the individual models inherit their parameters from the central weight repository and are then finetuned and ensembled in order to create a final system. We evaluate our system on the CIFAR-10 classification dataset and demonstrate improved results over the single global best model suggested by the optimisation process, with a minor increase in resources required by the finetuning process. Our system achieves an error rate of 4.27% on the CIFAR-10 image classification task with only 36 hours of combined optimisation and training on a single NVIDIA GTX 1080Ti GPU.

Item Type: Conference or Workshop Item (Paper)
Uncontrolled Keywords: Deep Learning, Evolutionary Computation, Image Classification, Convolutional Neural Networks, Particle Swarm Optimisation
Subjects: G400 Computer Science
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Related URLs:
Depositing User: Paul Burns
Date Deposited: 28 Aug 2019 15:08
Last Modified: 01 Aug 2021 10:37
URI: http://nrl.northumbria.ac.uk/id/eprint/40456

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics