Decoder Choice Network for Metalearning

Liu, Jialin, Chao, Fei, Yang, Longzhi, Lin, Chih-Min, Shang, Changjing and Shen, Qiang (2021) Decoder Choice Network for Metalearning. IEEE Transactions on Cybernetics. pp. 1-14. ISSN 2168-2267 (In Press)

[img]
Preview
Text
2021-DCN20210716-q.pdf - Accepted Version

Download (814kB) | Preview
Official URL: https://doi.org/10.1109/TCYB.2021.3123403

Abstract

Metalearning has been widely applied for implementing few-shot learning and fast model adaptation. Particularly, existing metalearning methods have been exploited to learn the control mechanism for gradient descent processes, in an effort to facilitate gradient-based learning in gaining high speed and generalization ability. This article presents a novel method that controls the gradient descent process of the model parameters in a neural network, by limiting the model parameters within a low-dimensional latent space. The main challenge for implementing this idea is that a decoder with many parameters may be required. To tackle this problem, the article provides an alternative design of the decoder with a structure that shares certain weights, thereby reducing the number of required parameters. In addition, this work combines ensemble learning with the proposed approach to improve the overall learning performance. Systematic experimental studies demonstrate that the proposed approach offers results superior to the state of the art in performing the Omniglot classification and miniImageNet classification tasks.

Item Type: Article
Additional Information: Funding information: This work was supported in part by the National Natural Science Foundation of China under Grant 61673322, Grant 61673326, and Grant 91746103; in part by the Fundamental Research Funds for the Central Universities under Grant 20720190142; in part by the Key Project of National Key Research and Development Project under Grant 2017YFC1703303; in part by the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curie Grant under Agreement 663830; and in part by the Strategic Partner Acceleration Award through the Sêr Cymru II Programme, U.K., under Grant 80761-AU201.
Subjects: G400 Computer Science
G500 Information Systems
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Elena Carlaw
Date Deposited: 06 Jan 2022 10:05
Last Modified: 06 Jan 2022 10:15
URI: http://nrl.northumbria.ac.uk/id/eprint/48085

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics