Wearable Structured Mental-Sensing-Graph Measurement

Yang, Daili, Gao, Bin, Woo, Wai Lok, Wen, Houlai, Zhao, Yihong and Gao, Zhao (2022) Wearable Structured Mental-Sensing-Graph Measurement. IEEE Transactions on Instrumentation and Measurement. ISSN 0018-9456 (In Press)

Preview

Text Wearable_Structured_Mental-Sensing-Graph_Measurement.pdf - Accepted Version Download (3MB) | Preview

Abstract

The emotional assessment under internet of things (IoT) architecture can support researchers to establish the relationships between human social and physiological signals and emotions. In this paper, a wearable emotion sensing system is developed under narrow band internet of things (NB-IoT) wireless communication technology. The wearable sensing device integrates social linked sensors including voice, activity, and heart rate. Using this system, a dating experiment is set up to investigate multimodal factors of male’s attractiveness perception. In particular, the multimodal data are fused in a graph structure, and this further leads to a graph convolutional neural networks model for emotion evaluation and a mental-sensing-graph intelligent interpreter. Different types of mental-sensing-graphs are fused during the training stage, and the model achieves a verification accuracy of 0.93. The intrinsic relationships among the multimodal data have been captured by the subgraphs which have star-shaped structures, and the center of the subgraphs are mostly audio node. The obtained results show that the attractiveness perception of the male participants in dating is more aligned to language communication. The results also reveal that when the male participants date highly attractive women during the experiments, a significant correlation is observed between the multimodal features and the attractiveness perception levels.

Actions (login required)

View Item

Downloads