Students' Perceptions of Characteristics of Effective College Teachers: A Validity Study of a Teaching Evaluation Form Using a Mixed-Methods Analysis

• Published in: American educational research journal 2007-03, Vol.44 (1), p.113-160
• Main Authors: Onwuegbuzie, Anthony J., Witcher, Ann E., Collins, Kathleen M. T., Filer, Janet D., Wiedmaier, Cheryl D., Moore, Chris W.
• Format: Article
• Language: English
• Subjects: Accountability; Classroom Communication; College Faculty; College Instruction; College students; Decision making; Education; Educational Research; Enthusiasm; Evaluation Methods; Evidence; Faculty Development; Graduate Students; Investigations; Learning; Majors (Students); Mixed methods research; Preservice teachers; R&D; Ratings & rankings; Research & development; Student Attitudes; Student Empowerment; Student Evaluation of Teacher Performance; Teacher Behavior; Teacher Characteristics; Teacher Effectiveness; Teacher Evaluation; Teacher evaluations; Teacher Responsibility; Teacher Student Relationship; Teachers; Teaching; Teaching assistants; Teaching Models; Teaching, Learning and Human Development; Test Construction; Thematic analysis; Undergraduate Students; Universities; Validation studies; Validity
• ISSN: 0002-8312; 1935-1011
• DOI: 10.3102/0002831206298169

This study used a multistage mixed-methods analysis to assess the content-related validity (i.e., item validity, sampling validity) and construct-related validity (i.e., substantive validity, structural validity, outcome validity, generalizability) of a teaching evaluation form (TEF) by examining students'perceptions of characteristics of effective college teachers. Participants were 912 undergraduate and graduate students (10.7% of student body) from various academic majors enrolled at a public university. A sequential mixed-methods analysis led to the development of the CARE-RESPECTED Model of Teaching Evaluation, which represented characteristics that students considered to reflect effective college teaching-comprising four meta-themes (communicator, advocate, responsible, empowering) and nine themes (responsive, enthusiast, student centered, professional, expert, connector, transmitter, ethical, and director). Three of the most prevalent themes were not represented by any of the TEF items; also, endorsement of most themes varied by student attribute (e.g., gender, age), calling into question the content-and construct-related validity of the TEF scores.