Learning a good representation with unsymmetrical auto-encoder

Sun, Yanan, Mao, Hua, Guo, Quan and Yi, Zhang (2016) Learning a good representation with unsymmetrical auto-encoder. Neural Computing and Applications, 27 (5). pp. 1361-1367. ISSN 0941-0643

Sun et al - Learning a good representation with unsymmetrical auto-encoder AAM.pdf - Accepted Version

Download (345kB) | Preview
Official URL: http://dx.doi.org/10.1007/s00521-015-1939-3


Auto-encoders play a fundamental role in unsupervised feature learning and learning initial parameters of deep architectures for supervised tasks. For given input samples, robust features are used to generate robust representations from two perspectives: (1) invariant to small variation of samples and (2) reconstruction by decoders with minimal error. Traditional auto-encoders with different regularization terms have symmetrical numbers of encoder and decoder layers, and sometimes parameters. We investigate the relation between the number of layers and propose an unsymmetrical structure, i.e., an unsymmetrical auto-encoder (UAE), to learn more effective features. We present empirical results of feature learning using the UAE and state-of-the-art auto-encoders for classification tasks with a range of datasets. We also analyze the gradient vanishing problem mathematically and provide suggestions for the appropriate number of layers to use in UAEs with a logistic activation function. In our experiments, UAEs demonstrated superior performance with the same configuration compared to other auto-encoders.

Item Type: Article
Uncontrolled Keywords: Auto-encoder, Neural networks, Feature learning, Deep learning, Unsupervised learning
Subjects: G400 Computer Science
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Paul Burns
Date Deposited: 13 Jun 2019 14:59
Last Modified: 01 Aug 2021 11:30
URI: http://nrl.northumbria.ac.uk/id/eprint/39680

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics