Detecting distracted students in educational VR environments using machine learning on eye gaze data

• Published in: Computers & graphics 2022-12, Vol.109, p.75-87
• Main Authors: Asish, Sarker Monojit, Kulshreshth, Arun K., Borst, Christoph W.
• Format: Article
• Language: English
• Subjects: Deep Learning; Distraction Detection; Education; Eye Tracking; Machine Learning; Virtual Reality
• ISSN: 0097-8493; 1873-7684
• DOI: 10.1016/j.cag.2022.10.007

Virtual Reality (VR) has been found useful to improve engagement and retention level of students, for some topics, compared to traditional learning tools such as books, and videos. However, a student could still get distracted and disengaged due to a variety of factors including stress, mind-wandering, unwanted noise, and external alerts. Student eye gaze data could be useful for detecting these distracted students. Gaze data-based visualizations have been proposed in the past to help a teacher monitor distracted students. However, it is not practical for a teacher to monitor a large number of student indicators while teaching. To help filter students based on distraction level, we propose an automated system based on machine learning to classify students based on their distraction level. The key aspects are: (1) we created a labeled eye gaze dataset from an educational VR environment, (2) we propose an automatic system to gauge a student’s distraction level from gaze data, and (3) we apply and compare several classifiers for this purpose. Each classifier classifies distraction, per educational activity section, into one of three levels (low, mid or high). Our results show that Random Forest (RF) classifier had the best accuracy (98.88%) compared to the other models we tested. Additionally, a personalized machine learning model using either RF, kNN, or Extreme Gradient Boosting (XGBoost) model was found to improve the classification accuracy significantly. [Display omitted] •Detecting distracted students in VR with machine learning using eye gaze data.•Developed data labeling and verification algorithm for eye gaze data.•Designed an educational VR environment with different instructional components.•Compared several machine learning algorithms for distraction classification.•Explored the efficacy of personalized machine learning models for classification.