Evolving Deep Architecture Generation with Residual Connections for Image Classification Using Particle Swarm Optimization

Lawrence, Tom, Zhang, Li, Rogage, Kay and Lim, Chee Peng (2021) Evolving Deep Architecture Generation with Residual Connections for Image Classification Using Particle Swarm Optimization. Sensors, 21 (23). p. 7936. ISSN 1424-8220

sensors-21-07936.pdf - Published Version
Available under License Creative Commons Attribution 4.0.

Download (773kB) | Preview
Official URL: https://doi.org/10.3390/s21237936


Automated deep neural architecture generation has gained increasing attention. However, exiting studies either optimize important design choices, without taking advantage of modern strategies such as residual/dense connections, or they optimize residual/dense networks but reduce search space by eliminating fine-grained network setting choices. To address the aforementioned weaknesses, we propose a novel particle swarm optimization (PSO)-based deep architecture generation algorithm, to devise deep networks with residual connections, whilst performing a thorough search which optimizes important design choices. A PSO variant is proposed which incorporates a new encoding scheme and a new search mechanism guided by non-uniformly randomly selected neighboring and global promising solutions for the search of optimal architectures. Specifically, the proposed encoding scheme is able to describe convolutional neural network architecture configurations with residual connections. Evaluated using benchmark datasets, the proposed model outperforms existing state-of-the-art methods for architecture generation. Owing to the guidance of diverse non-uniformly selected neighboring promising solutions in combination with the swarm leader at fine-grained and global levels, the proposed model produces a rich assortment of residual architectures with great diversity. Our devised networks show better capabilities in tackling vanishing gradients with up to 4.34 improvement of mean accuracy in comparison with those of existing studies.

Item Type: Article
Additional Information: Funding information: This work was supported by the European Regional Development Fund—Industrial Intensive Innovation Programme.
Uncontrolled Keywords: deep architecture generation; deep residual network; particle swarm optimization; image classification
Subjects: G400 Computer Science
G500 Information Systems
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Elena Carlaw
Date Deposited: 29 Nov 2021 14:55
Last Modified: 29 Nov 2021 15:00
URI: http://nrl.northumbria.ac.uk/id/eprint/47853

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics