Unobtrusive Behavioral Analysis of Students in Classroom Environment Using Non-Verbal Cues

• Published in: IEEE access 2019, Vol.7, p.150693-150709
• Main Authors: Ashwin, T. S., Guddeti, Ram Mohana Reddy
• Format: Article
• Language: English
• Subjects: affect sensing and analysis; Affective computing; Annotations; Artificial neural networks; behavioral patterns; classroom data in the wild; Classrooms; Computer architecture; Electronic learning; Emotion recognition; Face recognition; Image classification; Image recognition; Learning; multimodal analysis; Sensors; student engagement; Students
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2947519

Pervasive intelligent learning environments can be made more personalized by adapting the teaching strategies according to the students' emotional and behavioral engagements. The students' engagement analysis helps to foster those emotions and behavioral patterns that are beneficial to learning, thus improving the effectiveness of the teaching-learning process. Unobtrusive student engagement analysis is performed using the students' non-verbal cues such as facial expressions, hand gestures, and body postures. Though there exist several techniques for classifying the engagement of a single student present in a single image frame, there are limited works on the students' engagement analysis in a classroom environment. In this paper, we propose a convolutional neural network architecture for unobtrusive students' engagement analysis using non-verbal cues. The proposed architecture is trained and tested on faces, hand gestures and body postures in the wild of more than 350 students present in a classroom environment, with each test image containing multiple students in a single image frame. The data annotation is performed using the gold standard study, and the annotators reliably agree with Cohen's \kappa = 0.43 . We obtained 71% accuracy for the students' engagement level classification. Further, a pre-test/post-test analysis was performed, and it was observed that there is a positive correlation between the students' engagement and their test performance.