Quantitative analysis of the rubric as an assessment tool: an empirical study of student peer- group rating

• Published in: International journal of science education 2003-12, Vol.25 (12), p.1509-1528
• Main Authors: Hafner, John, Hafner, Patti
• Format: Article
• Language: English
• Subjects: Biologie; Biology; College Students; Curriculum subjects: programmes and methods; Educational sciences; Empirische Untersuchung; Evaluation Methods; Exact and natural sciences; Generalizability Theory; Interrater Reliability; Peer Evaluation; Peer Group; Scoring Rubrics; Self Evaluation (Individuals); Student; Teaching methods; Test Reliability; Test Validity
• ISSN: 0950-0693; 1464-5289
• DOI: 10.1080/0950069022000038268

Although the rubric has emerged as one of the most popular assessment tools in progressive educational programs, there is an unfortunate dearth of information in the literature quantifying the actual effectiveness of the rubric as an assessment tool in the hands of the students. This study focuses on the validity and reliability of the rubric as an assessment tool for student peer-group evaluation in an effort to further explore the use and effectiveness of the rubric. A total of 1577 peer-group ratings using a rubric for an oral presentation was used in this 3-year study involving 107 college biology students. A quantitative analysis of the rubric used in this study shows that it is used consistently by both students and the instructor across the study years. Moreover, the rubric appears to be 'gender neutral' and the students' academic strength has no significant bearing on the way that they employ the rubric. A significant, one-to-one relationship (slope = 1.0) between the instructor's assessment and the students' rating is seen across all years using the rubric. A generalizability study yields estimates of inter-rater reliability of moderate values across all years and allows for the estimation of variance components. Taken together, these data indicate that the general form and evaluative criteria of the rubric are clear and that the rubric is a useful assessment tool for peer-group (and self-) assessment by students. To our knowledge, these data provide the first statistical documentation of the validity and reliability of the rubric for student peer-group assessment.