Semi-feature level fusion for bimodal affect regression based on facial and bodily expressions

Zhang, Yang and Zhang, Li (2015) Semi-feature level fusion for bimodal affect regression based on facial and bodily expressions. In: AAMAS '15 Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS, pp. 1557-1565. ISBN 978-1-4503-3413-6

Full text not available from this repository. (Request a copy)

Abstract

Automatic emotion recognition has been widely studied and applied to various computer vision tasks (e.g. health monitoring, driver state surveillance, personalized learning, and security monitoring). As revealed by recent psychological and behavioral research, facial expressions are good in communicating categorical emotions (e.g. happy, sad, surprise, etc.), while bodily expressions could contribute more to the perception of dimensional emotional states (e.g. arousal and valence). In this paper, we propose a semi-feature level fusion framework that incorporates affective information of both the facial and bodily modalities to draw a more reliable interpretation of users’ emotional states in a valence–arousal space. The Genetic Algorithm is also applied to conduct automatic feature optimization. We subsequently propose an ensemble regression model to robustly predict users’ continuous affective dimensions in the valence–arousal space. The empirical findings indicate that by combining the optimal discriminative bodily features and the derived Action Unit intensities as inputs, the proposed system with adaptive ensemble regressors achieves the best performance for the regression of both the arousal and valence dimensions.

Item Type:	Book Section
Uncontrolled Keywords:	Affective computing, multimodal affect sensing, adaptive ensemble models, feature selection, optimization
Subjects:	G400 Computer Science G700 Artificial Intelligence
Department:	Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User:	Users 6424 not found.
Date Deposited:	03 Nov 2015 15:01
Last Modified:	12 Oct 2019 20:51
URI:	http://nrl.northumbria.ac.uk/id/eprint/24287

Downloads