Gaze-Informed Egocentric Action Recognition for Memory Aid Systems

Zuo, Zheming, Yang, Longzhi, Peng, Yonghong, Chao, Fei and Qu, Yanpeng (2018) Gaze-Informed Egocentric Action Recognition for Memory Aid Systems. IEEE Access, 6. pp. 12894-12904. ISSN 2169-3536

08305456.pdf - Published Version
Available under License Creative Commons Attribution 4.0.

Download (14MB) | Preview
Official URL:


Egocentric action recognition has been intensively studied in the fields of computer vision and clinical science with applications in pervasive health-care. The majority of the existing egocentric action recognition techniques utilize the features extracted from either the entire contents or the regions of interest in video frames as the inputs of action classifiers. The former might suffer from moving backgrounds or irrelevant foregrounds usually associated with egocentric action videos, while the latter may be impaired by the mismatch between the calculated and the ground truth regions of interest. This paper proposes a new gaze-informed feature extraction approach, by which the features are extracted from the regions around the gaze points and thus representing the genuine regions of interest from a first person of view. The activity of daily life can then be classified based only on the identified regions using the extracted gaze-informed features. The proposed approach has been further applied to a memory support system for people with poor memory, such as those with Amnesia or dementia, and their carers. The experimental results demonstrate the efficacy of the proposed approach in egocentric action recognition and thus the potential of the memory support tool in health care.

Item Type: Article
Subjects: G400 Computer Science
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Paul Burns
Date Deposited: 22 Feb 2018 09:58
Last Modified: 01 Aug 2021 08:50

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics