Analysing students' self‐assessment practice in a distance education environment: Student behaviour, accuracy, and task‐related characteristics

• Published in: Journal of computer assisted learning 2024-04, Vol.40 (2), p.654-666
• Main Authors: Radović, Slaviša, Seidel, Niels, Haake, Joerg M., Kasakowskij, Regina
• Format: Article
• Language: English
• Subjects: Accuracy; CAI; College students; Computer assisted instruction; computer‐assisted feedback; criteria‐based self‐assessment; Distance learning; Evaluation; Iterative methods; Learning Processes; Lifelong learning; Questions; Replication; self‐assessment; self‐assessment accuracy; Students; Task complexity; Undergraduate Students; Undergraduate study
• ISSN: 0266-4909; 1365-2729
• DOI: 10.1111/jcal.12907

Background Self‐assessment serves to improve learning through timely feedback on one's solution and iterative refinement as a way to improve one's competence. However, the complexity of the self‐assessment process is widely recognized, as well as that students can benefit from it only if their assessment is accurate enough. Objectives In order to gain more insight into the self‐assessment process we analysed students' behaviour, accuracy, and question‐related characteristics that influence the capability of self‐assessment in two studies. Methods The initial study examined 131 undergraduate students using voluntary self‐assessment questions in an online course in a B.Sc. Computer Science program while a year later a replication study with the same research settings was applied to a different cohort of 264 undergraduate students with minor modifications to the question design, in the light of the original findings. Results and Conclusions Results from both studies show that similar patterns could be observed for usage and of accuracy and score distribution for almost all questions. Item difficulty and comprehensiveness of the sample solution were identified as features of self‐assessment questions affecting student's self‐assessment capability. The replication study showed that task design can be modified to affect students' accuracy. Recommendations to make self‐assessment tasks effective and efficient for learning are provided. Lay Description What is already known about this topic Students should be given the opportunity to self‐assess their knowledge during the learning process. Self‐assessment can increase students' motivation to learn and promotes academic achievement and self‐efficacy. What this paper adds Both, the initial and replication, studies provide better understanding of learners' behaviour and task design characteristics to afford better learning support and maximize the potential for desirable outcomes. Task design (task complexity, scope and transfer requirements as well as quantitative measures such as text length and number of evaluation criteria) can be modified to affect students' accuracy. Implications for practice Supporting students learning is not a one‐off activity, but part of a curriculum improvement and lifelong learning initiative, therefore we recommend that self‐assessment tasks are continuously monitored and, if necessary, improved to suit the needs of learners in a specific context. Self‐assessment should become a habit o