A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect

Hu, Shanfeng, Bhattacharya, Hindol, Chattopadhyay, Matangini, Aslam, Nauman and Shum, Hubert P. H. (2019) A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect. In: 2018 12th International Conference on Software, Knowledge, Information Management & Applications (SKIMA): Phnom Penh, Cambodia 3 – 5 December 2018. International Conference on Software, Knowledge, Information Management and Applications (SKIMA), Proceedings . IEEE, Piscataway, NJ, pp. 1-8. ISBN 9781538691427, 9781538691410

Text (Full text)
Hu et al - A Dual-Stream Recurrent Neural Network AAM.pdf - Accepted Version

Download (503kB) | Preview
Official URL: http://doi.org/10.1109/SKIMA.2018.8631537


Convenience internet access and ubiquitous computing have opened up new avenues for learning and teaching. They are now no longer confined to the classroom walls, but are available to anyone connected to the internet. E-learning has opened massive opportunities for learners who otherwise would have been constrained due to geographical distances, time and/or cost factors. It has revolutionized the learning methods and represents a paradigm shift from traditional learning methods. However, despite all its advantages, e-learning is not without its own shortcomings. Understanding the effectiveness of a teaching strategy through learner feedback has been a key performance measure and decision making criteria to fine tune the teaching strategy. However, traditional methods of collecting learner feedback are inadequate in a geographically distributed, virtual setup of the e-learning environment. Innovative and novel learner feedback collection mechanism is hence the need of the hour. In this work, we design and develop a deep learning based student feedback prediction system by recognizing the subtle facial motions during a student’s learning activity. This allows the system to infer the needs of the learners as if it is a real human teacher in order to provide the appropriate feedback. We propose a recurrent convolutional neural network structure to understand the color and depth streams of video taken by an RGB-D camera. Experimental results have shown that our system achieve high accuracy in estimating the feedback labels. While we demonstrate the proposed framework in an e-learning setup, it can be adapted to other applications such as in-house patient monitoring and rehabilitation training.

Item Type: Book Section
Uncontrolled Keywords: feedback prediction, Kinect, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep learning, e-Learning
Subjects: G400 Computer Science
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Paul Burns
Date Deposited: 02 Nov 2018 10:19
Last Modified: 01 Aug 2021 11:53
URI: http://nrl.northumbria.ac.uk/id/eprint/36473

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics