Automated detection of emotional and cognitive engagement in MOOC discussions to predict learning achievement

• Published in: Computers and education 2022-05, Vol.181, p.104461, Article 104461
• Main Authors: Liu, Sannyuya, Liu, Shiqi, Liu, Zhi, Peng, Xian, Yang, Zongkai
• Format: Article
• Language: English
• Subjects: 21st century abilities; Cooperative/collaborative learning; Data science applications in education; Distance education and online learning; Evaluation methodologies
• ISSN: 0360-1315; 1873-782X
• DOI: 10.1016/j.compedu.2022.104461

In the MOOC forum discussions, emotional and cognitive engagement are two prominent aspects of learning engagement. Moreover, emotional and cognitive engagement have an interactive relationship and can jointly predict learning achievement. However, these interwoven relationships have not been thoroughly explored. Furthermore, the limitations on detection methods for emotional and cognitive engagement have hindered the practice and theory progress. This study aimed to develop a novel text classification model to automatically detect emotional and cognitive engagement and investigate their complex relationships with achievement, which are beneficial for improving learning engagement and historically low completion rates of MOOCs. Firstly, this study proposed a robust and interpretable NLP model called the bidirectional encoder representation from the transformers-convolutional neural network (BERT-CNN). Compared with models in previous studies, it improved the F1 values of emotional and cognitive engagement recognition tasks by 10% and 8%, respectively. Secondly, this study used BERT-CNN to analyze 8867 learners’ discussions in a MOOC forum. Structural equation modeling indicated that emotional and cognitive engagement have an interactive relationship and a combined effect on learning achievement. Specifically, positive and confused emotions contributed more to higher-level cognition than negative emotions. Co-occurring emotion and cognition indicators jointly predicted learning achievement with higher reliability. In summary, this study has significant methodological implications for the automated measurement of emotional and cognitive engagement. Moreover, the study revealed the dominant role of emotional engagement on cognitive engagement and provided suggestions for improving MOOC learners' achievement. •A deep learning model for detecting emotional and cognitive engagement is developed.•The model realizes interpretable and robust automatic discourse analysis.•The mathematic model of emotional, cognitive engagement, and achievement is validated.•Positive and confused emotions are associated with higher-order cognitive engagement.•Using emotional and cognitive engagement to predict achievement is more reliable.