A Criterion-Referenced Approach to Student Ratings of Instruction

We developed a criterion-referenced student rating of instruction (SRI) to facilitate formative assessment of teaching. It involves four dimensions of teaching quality that are grounded in current instructional design principles: Organization and structure, Assessment and feedback, Personal interact...

Full description

Saved in:
Bibliographic Details
Published in:Research in higher education 2017-08, Vol.58 (5), p.545-567
Main Authors: Meyer, J. Patrick, Doromal, Justin B., Wei, Xiaoxin, Zhu, Shi
Format: Article
Language:English
Subjects:
Course Evaluation
Criterion Referenced Tests
Data Interpretation
Education
Educational Quality
Evaluation Methods
Formative Evaluation
Gender Differences
Higher Education
Instructional Design
Instructional Effectiveness
Instructional Improvement
Item Response Theory
Predictor Variables
Rating Scales
Scores
Student Evaluation
Student Evaluation of Teacher Performance
Teacher Characteristics
Teacher Effectiveness
Teacher evaluations
Teaching Methods
Teaching Styles
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We developed a criterion-referenced student rating of instruction (SRI) to facilitate formative assessment of teaching. It involves four dimensions of teaching quality that are grounded in current instructional design principles: Organization and structure, Assessment and feedback, Personal interactions, and Academic rigor. Using item response theory and Wright mapping methods, we describe teaching characteristics at various points along the latent continuum for each scale. These maps enable criterion-referenced score interpretation by making an explicit connection between test performance and the theoretical framework. We explain the way our Wright maps can be used to enhance an instructor's ability to interpret scores and identify ways to refine teaching. Although our work is aimed at improving score interpretation, a criterion-referenced test is not immune to factors that may bias test scores. The literature on SRIs is filled with research on factors unrelated to teaching that may bias scores. Therefore, we also used multilevel models to evaluate the extent to which student and course characteristic may affect scores and compromise score interpretation. Results indicated that student anger and the interaction between student gender and instructor gender are significant effects that account for a small amount of variance in SRI scores. All things considered, our criterion-referenced approach to SRIs is a viable way to describe teaching quality and help instructors refine pedagogy and facilitate course development.
ISSN:0361-0365
1573-188X
DOI:10.1007/s11162-016-9437-8