Improving marking reliability of scientific writing with the Developing Understanding of Assessment for Learning programme

• Published in: Assessment and evaluation in higher education 2013-08, Vol.38 (5), p.536-553
• Main Authors: Bird, Fiona L., Yucel, Robyn
• Format: Article
• Language: English
• Subjects: Academic achievement; Academic language; Australia; Biology; Collaboration; College Faculty; College Freshmen; College Science; Communities of Practice; Content Area Writing; Developing Understanding of Assessment for Learning (DUAL) Program; Educational evaluation; Efficiency; Feedback; Feedback (Response); Foreign Countries; Grading; Higher education; inter-rater reliability; Interrater Reliability; marking efficiency; Moderation (Assessment); moderation of assessment; scientific writing; Scoring Rubrics; Self evaluation; Student assessment; Student Evaluation; Students; Technical writing; Training
• ISSN: 0260-2938; 1469-297X
• DOI: 10.1080/02602938.2012.658155

The Developing Understanding of Assessment for Learning (DUAL) programme was developed with the dual aims of improving both the quality and consistency of feedback students receive and the students' ability to use that feedback to improve. DUAL comprises a range of processes (including marking rubrics, sample reports, moderation discussions and peer-review activities) which support explicit knowledge exchange between staff, and between staff and students and the development of tacit knowledge of standards in all participants. This paper describes the study which quantified the extent to which the DUAL programme improved reliability and efficiency of marking of first year biology reports by a large team of laboratory demonstrators. Marker reliability increased after participation in DUAL, with demonstrators marking laboratory reports significantly closer to the marks awarded by an expert marker of the discipline. Efficiency was also improved since time taken to mark the reports did not increase significantly, despite the fact that demonstrators were adapting to a new system of marking. Demonstrators valued the programme and nominated the marking rubric, feedback code and moderation discussion as the most useful elements. The community of practice built through joint participation in the DUAL programme created a supportive working environment for demonstrators and strengthened the collaboration between the discipline and academic language and learning lecturers.