Wearable Structured Mental-Sensing-Graph Measurement

Yang, Daili, Gao, Bin, Woo, Wai Lok, Wen, Houlai, Zhao, Yihong and Gao, Zhao (2022) Wearable Structured Mental-Sensing-Graph Measurement. IEEE Transactions on Instrumentation and Measurement. ISSN 0018-9456 (In Press)

Wearable_Structured_Mental-Sensing-Graph_Measurement.pdf - Accepted Version

Download (3MB) | Preview
Official URL: https://doi.org/10.1109/TIM.2022.3232163


The emotional assessment under internet of things (IoT) architecture can support researchers to establish the relationships between human social and physiological signals and emotions. In this paper, a wearable emotion sensing system is developed under narrow band internet of things (NB-IoT) wireless communication technology. The wearable sensing device integrates social linked sensors including voice, activity, and heart rate. Using this system, a dating experiment is set up to investigate multimodal factors of male’s attractiveness perception. In particular, the multimodal data are fused in a graph structure, and this further leads to a graph convolutional neural networks model for emotion evaluation and a mental-sensing-graph intelligent interpreter. Different types of mental-sensing-graphs are fused during the training stage, and the model achieves a verification accuracy of 0.93. The intrinsic relationships among the multimodal data have been captured by the subgraphs which have star-shaped structures, and the center of the subgraphs are mostly audio node. The obtained results show that the attractiveness perception of the male participants in dating is more aligned to language communication. The results also reveal that when the male participants date highly attractive women during the experiments, a significant correlation is observed between the multimodal features and the attractiveness perception levels.

Item Type: Article
Uncontrolled Keywords: Biomedical monitoring, Classification and explanation, Emotional assessment, Feature extraction, Heart rate, Internet of things, Mental-Sensing-Graph, NB-IoT, Sensors, Subgraphs, Temperature sensors, Wearable computers, Wearable device, Wearable sensors
Subjects: G400 Computer Science
G900 Others in Mathematical and Computing Sciences
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Rachel Branson
Date Deposited: 31 Jan 2023 15:32
Last Modified: 01 Feb 2023 09:16
URI: https://nrl.northumbria.ac.uk/id/eprint/51285

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics