A Deep Learning Based Wearable Healthcare Iot Device for AI-Enabled Hearing Assistance Automation

Young, Fraser, Zhang, Li, Jiang, Richard, Liu, Han and Wall, Conor (2020) A Deep Learning Based Wearable Healthcare Iot Device for AI-Enabled Hearing Assistance Automation. In: Proceedings of 2020 International Conference on Machine Learning and Cybernetics, ICMLC 2020. IEEE, Piscataway, NJ, pp. 235-240. ISBN 9780738124261

AAM.pdf - Accepted Version

Download (736kB) | Preview
Official URL: https://doi.org/10.1109/ICMLC51923.2020.9469537


With the recent booming of artificial intelligence (AI), particularly deep learning techniques, digital healthcare is one of the prevalent areas that could gain benefits from AI-enabled functionality. This research presents a novel AI-enabled Internet of Things (IoT) device operating from the ESP-8266 platform capable of assisting those who suffer from impairment of hearing or deafness to communicate with others in conversations. In the proposed solution, a server application is created that leverages Google's online speech recognition service to convert the received conversations into texts, then deployed to a micro-display attached to the glasses to display the conversation contents to deaf people, to enable and assist conversation as normal with the general population. Furthermore, in order to raise alert of traffic or dangerous scenarios, an 'urban-emergency' classifier is developed using a deep learning model, Inception-v4, with transfer learning to detect/recognize alerting/alarming sounds, such as a horn sound or a fire alarm, with texts generated to alert the prospective user. The training of Inception-v4 was carried out on a consumer desktop PC and then implemented into the AI-based IoT application. The empirical results indicate that the developed prototype system achieves an accuracy rate of 92% for sound recognition and classification with real-time performance.

Item Type: Book Section
Additional Information: 19th International Conference on Machine Learning and Cybernetics, ICMLC 2020 4/12/20 → … Virtual, Online
Uncontrolled Keywords: Convolutional Neural Network, Deafness Assistance, Deep Learning, Healthcare, Internet of Things, Wearables
Subjects: G400 Computer Science
G900 Others in Mathematical and Computing Sciences
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Rachel Branson
Date Deposited: 04 May 2022 13:58
Last Modified: 04 May 2022 14:00
URI: http://nrl.northumbria.ac.uk/id/eprint/49034

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics