Using multimodal learning analytics to validate digital traces of self-regulated learning in a laboratory study and predict performance in undergraduate courses

• Published in: Journal of educational psychology 2024-10
• Main Authors: Bernacki, Matthew L., Yu, Linyu, Kuhlmann, Shelbi L., Plumley, Robert D., Greene, Jeffrey A., Duke, Rebekah F., Freed, Rebekah, Hollander-Blackmon, Christina, Hogan, Kelly A.
• Format: Article
• Language: English
• Subjects: College Students; Digital Technology; Female; Human; Learning Strategies; Male; Self-Regulated Learning; STEM
• ISSN: 0022-0663; 1939-2176
• DOI: 10.1037/edu0000890

Undergraduates enrolled in large, active learning courses must self-regulate their learning (self-regulated learning [SRL]) by appraising tasks, making plans, setting goals, and enacting and monitoring strategies. SRL researchers have relied on self-report and learner-mediated methods during academic tasks studied in laboratories and now collect digital event data when learners engage with technology-based tools in classrooms. Inferring SRL processes from digital events and testing their validity is challenging. We aligned digital and verbal SRL event data to validate digital events as traces of SRL and used them to predict achievement in lab and course settings. In Study 1, we sampled a learning task from a biology course into a laboratory setting. Enrolled students ( N = 48) completed the lesson using digital resources (e.g., online textbook, course site) while thinking aloud weeks before it was taught in class. Analyses confirmed that 10 digital events reliably co-occurred ≥ 70% of the time with verbalized task definition and strategy use macroprocesses. Some digital events co-occurred with multiple verbalized SRL macroprocesses. Variance in occurrence of validated digital events was limited in lab sessions, and they explained statistically nonsignificant variance in learners’ performance on lesson quizzes. In Study 2, lesson-specific digital event data from learners ( N = 307) enrolled in the course (but not in Study 1) predicted performance on lesson-specific exam items, final exams, and course grades. Validated digital events also predicted final exam and course grades in the next semester ( N = 432). Digital events can be validated to reflect SRL processes and scaled to explain achievement in naturalistic undergraduate education settings. (PsycInfo Database Record (c) 2024 APA, all rights reserved) (Source: journal abstract)