An Assessment of Reliability and Validity of a Rubric for Grading APA-Style Introductions

This article describes the empirical evaluation of the reliability and validity of a grading rubric for grading APA-style introductions of undergraduate students. Levels of interrater agreement and intrarater agreement were not extremely high but were similar to values reported in the literature for...

Full description

Saved in:
Bibliographic Details
Published in:Teaching of psychology 2009-04, Vol.36 (2), p.102-107
Main Authors: Stellmack, Mark A., Konheim-Kalkstein, Yasmine L., Manor, Julia E., Massey, Abigail R., Schmitz, Julie Ann P.
Format: Article
Language:English
Subjects:
Grading
Guides
Instructional Effectiveness
Interrater Reliability
Professional Associations
Psychology
Scoring Rubrics
Undergraduate Students
Validity
Writing (Composition)
Writing for Publication
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article describes the empirical evaluation of the reliability and validity of a grading rubric for grading APA-style introductions of undergraduate students. Levels of interrater agreement and intrarater agreement were not extremely high but were similar to values reported in the literature for comparably structured rubrics. Rank-order correlations between graders who used the rubric and an experienced instructor who ranked the papers separately and holistically provided evidence for the rubric's validity. Although this rubric has utility as an instructional tool, the data underscore the seemingly unavoidable subjectivity inherent in grading student writing. Instructors are cautioned that merely using an explicit, carefully developed rubric does not guarantee high reliability.
ISSN:0098-6283
1532-8023
DOI:10.1080/00986280902739776