An Efficient Threshold-Driven Aggregate-Label Learning Algorithm for Multimodal Information Processing

Zhang, Malu, Luo, Xiaoling, Wu, Jibin, Chen, Yi, Belatreche, Ammar, Pan, Zihan, Qu, Hong and Li, Haizhou (2020) An Efficient Threshold-Driven Aggregate-Label Learning Algorithm for Multimodal Information Processing. IEEE Journal of Selected Topics in Signal Processing, 14 (3). pp. 592-602. ISSN 1932-4553

A Belatreche_IEEE JSTSP_An Efficient Threshold-Driven Aggregate-Label Learning Algorithm for Multimodal Information Processing.pdf - Accepted Version

Download (1MB) | Preview
Official URL:


The aggregate-label learning paradigm tackles the long-standing temporary credit assignment (TCA) problem in neuroscience and machine learning, enabling spiking neural networks to learn multimodal sensory clues with delayed feedback signals. However, the existing aggregate-label learning algorithms only work for single spiking neurons, and with low learning efficiency, which limit their real-world applicability. To address these limitations, we first propose an efficient threshold-driven plasticity algorithm for spiking neurons, namely ETDP. It enables spiking neurons to generate the desired number of spikes that match the magnitude of delayed feedback signals and to learn useful multimodal sensory clues embedded within spontaneous spiking activities. Furthermore, we extend the ETDP algorithm to support multi-layer spiking neural networks (SNNs), which significantly improves the applicability of aggregate-label learning algorithms. We also validate the multi-layer ETDP learning algorithm in a multimodal computation framework for audio-visual pattern recognition. Experimental results on both synthetic and realistic datasets show significant improvements in the learning efficiency and model capacity over the existing aggregate-label learning algorithms. It, therefore, provides many opportunities for solving real-world multimodal pattern recognition tasks with spiking neural networks.

Item Type: Article
Uncontrolled Keywords: Spiking neurons, spiking neural networks, aggregate-label learning, synaptic plasticity, multimodal information
Subjects: G400 Computer Science
G500 Information Systems
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Elena Carlaw
Date Deposited: 27 Apr 2020 10:01
Last Modified: 31 Jul 2021 11:47

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics