A Self-Regulated Learning Analytics Prediction-and-Intervention Design: Detecting and Supporting Struggling Biology Students

• Published in: Journal of educational psychology 2022-11, Vol.114 (8), p.1801-1816
• Main Authors: Cogliano, MeganClaire, Bernacki, Matthew L., Hilpert, Jonathan C., Strong, Christy L.
• Format: Article
• Language: English
• Subjects: Academic Achievement; Algorithms; At Risk Students; Biology; College Science; College Students; Curriculum; Digital Interventions; Educational Technology; Female; Grades (Scholastic); Human; Learning Analytics; Learning Management Systems; Male; Management Systems; Prediction; Predictions; Prior Learning; Program Effectiveness; School Based Intervention; Science Achievement; Science education; Self Management; Self regulation; Self-Regulated Learning; Skill Development; Training; Undergraduate Students
• ISSN: 0022-0663; 1939-2176
• DOI: 10.1037/edu0000745

We investigated the effects of a learning analytics-driven prediction modeling platform and a brief digital self-regulated learning skill training program targeted to support undergraduate biology students identified as likely to perform poorly in the course. A prediction model comprising prior knowledge scores and learning management system log data of student activities during the first 2 weeks in the course was applied to flag students who were likely to earn a C or worse (N = 143). Students who were flagged were randomized into a flagged treatment (N = 79) or flagged control (N = 64) condition. We found that training students who were flagged as likely to perform poorly significantly improved their achievement on unit exams, compared with students who were also flagged but did not receive the training. The effect of training on final examination was mediated by unit exam achievement. In addition, the students who were predicted to perform well (N = 83) and flagged treatment groups did not differ statistically significantly on academic performance. Training also had a significant effect on final course performance with students in the flagged treatment and nonflagged groups outperforming the flagged control students. The results indicate that an algorithm that uses behavioral data to predict achievement does so with sufficient accuracy to detect the large differences in achievement earned by two groups of learners distinguishable by their early, digital learning behaviors, and that a brief ∼15-minute digital skills training was sufficient to ameliorate these achievement differences when deployed before the first unit exam. Educational Impact and Implications When STEM (science, technology, engineering, and mathematics) undergraduates access resources like readings, notes, study guides, and quizzes on the course sites that accompany their lectures, they produce digital traces of learning events. These learning events can be interpreted based on the cognitive and metacognitive processes that the resource affords, serve as proxies for students' internal self-regulated learning processes, and be used to predict the achievement self-regulated learning is known to produce. We used machine learning to build an algorithm based on these digital events that predicted students' exam performance, then provided feedback and training or an additional biology learning activity to those who were predicted to perform poorly, and who were open to receiving learning suppo