Using rasch measurement to score, evaluate, and improve examinations in an anatomy course

• Published in: Anatomical sciences education 2014-11, Vol.7 (6), p.450-460
• Main Authors: Royal, Kenneth D., Gilliland, Kurt O., Kernick, Edward T.
• Format: Article
• Language: English
• Subjects: Anatomy; Anatomy - education; Construct Validity; Educational Measurement - methods; evaluation; Evaluation Methods; Evaluation Research; examinations; gross anatomy education; High Stakes Tests; Higher Education; Item Response Theory; Measurement Techniques; Medical Education; Program Improvement; Program Validation; Psychometrics; rasch measurement; Robustness (Statistics); Scoring; Scoring Rubrics; Student Evaluation; students assessments; Test Theory; Testing; Testing Programs; Validity
• ISSN: 1935-9772; 1935-9780
• DOI: 10.1002/ase.1436

Any examination that involves moderate to high stakes implications for examinees should be psychometrically sound and legally defensible. Currently, there are two broad and competing families of test theories that are used to score examination data. The majority of instructors outside the high‐stakes testing arena rely on classical test theory (CTT) methods. However, advances in item response theory software have made the application of these techniques much more accessible to classroom instructors. The purpose of this research is to analyze a common medical school anatomy examination using both the traditional CTT scoring method and a Rasch measurement scoring method to determine which technique provides more robust findings, and which set of psychometric indicators will be more meaningful and useful for anatomists looking to improve the psychometric quality and functioning of their examinations. Results produced by the more robust and meaningful methodology will undergo a rigorous psychometric validation process to evaluate construct validity. Implications of these techniques and additional possibilities for advanced applications are also discussed. Anat Sci Educ 7: 450–460. © 2014 American Association of Anatomists.