ImageLM: Interpretable image-based learner modelling for classifying learners’ computational thinking

• Published in: Expert systems with applications 2024-03, Vol.238, p.122283, Article 122283
• Main Authors: Hooshyar, Danial, Yang, Yeongwook
• Format: Article
• Language: English
• Subjects: Computational thinking; Convolutional neural network; Image processing; Learner modelling; Trustworthy AI for education
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2023.122283

Predictive learner modelling is crucial for personalized education. While convolutional neural networks (CNNs) have shown great success in education, their potential in learner modelling via image data is unexplored. This research introduces a novel and interpretable approach for Image-based Learner Modelling (ImageLM) using CNNs and transfer learning to model learners’ performance and accordingly classify their computational thinking solutions. The approach integrates Grad-CAM, enabling it to provide insights into its decision-making process. Findings show that our custom CNN outperforms other models (namely ResNet, VGG, and Inception), with 83% accuracy in predicting solution correctness. More importantly, the ImageLM approach identifies the regions that contribute the most to the predictions, shedding light on learners' computational thinking knowledge and advancing toward trustworthy AI for education. These results underline the potential of utilizing imagery data from learners’ activities during the learning process to predict their performance, especially in challenging environments like programming where traditional feature extraction and learning might struggle.