Effects of Behavioral Anchors on Peer Evaluation Reliability

This paper presents comparisons of three peer evaluation instruments tested among students in undergraduate engineering classes: a single‐item instrument without behavioral anchors, a ten‐item instrument, and a single‐item behaviorally anchored instrument. Studies using the instruments in undergradu...

Full description

Saved in:
Bibliographic Details
Published in:Journal of engineering education (Washington, D.C.) D.C.), 2005-07, Vol.94 (3), p.319-326
Main Authors: Ohland, Matthew W., Layton, Richard A., Loughry, Misty L., Yuhasz, Amy G.
Format: Article
Language:English
Subjects:
assessment
Behavioral Objectives
behaviorally anchored rating scale
College students
Comparative studies
Educational evaluation
Engineering
Engineering Education
Peer Evaluation
Reliability
Student Behavior
Thinking Skills
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper presents comparisons of three peer evaluation instruments tested among students in undergraduate engineering classes: a single‐item instrument without behavioral anchors, a ten‐item instrument, and a single‐item behaviorally anchored instrument. Studies using the instruments in undergraduate engineering classes over four years show that the use of behavioral anchors significantly improves the inter‐rater reliability of the single‐item instrument. The inter‐rater reliability (based on four raters) of the behaviorally anchored instrument was 0.78, which was not significantly higher than that of the ten‐item instrument (0.74), but it was substantially more parsimonious. The results of this study add to the body of knowledge on evaluating students' performance in teams. This is critical since the ability to function in multidisciplinary teams is a required student learning outcome of engineering programs.
ISSN:1069-4730
2168-9830
DOI:10.1002/j.2168-9830.2005.tb00856.x