Evidence-based decision about test scoring rules in clinical anatomy multiple-choice examinations

• Published in: Anatomical sciences education 2015-05, Vol.8 (3), p.242-248
• Main Authors: Severo, Milton, Gaio, A. Rita, Povo, Ana, Silva-Pereira, Fernanda, Ferreira, Maria Amélia
• Format: Article
• Language: English
• Subjects: Adult; Anatomy; Anatomy - education; assessment of performance; Choice Behavior; Comparative Analysis; Correlation; Decision Making; Education, Medical, Undergraduate - methods; educational measurement; Educational Measurement - standards; Evidence; examinations; formula scoring; Humans; medical education; methodology; Multiple Choice Tests; number-right scoring; Reproducibility of Results; Research Design - standards; Scoring; Scoring Formulas; Students, Medical; Test Reliability
• ISSN: 1935-9772; 1935-9780
• DOI: 10.1002/ase.1478

In theory the formula scoring methods increase the reliability of multiple‐choice tests in comparison with number‐right scoring. This study aimed to evaluate the impact of the formula scoring method in clinical anatomy multiple‐choice examinations, and to compare it with that from the number‐right scoring method, hoping to achieve an evidence‐based decision about test scoring rules. Two hundred and ninety‐eight students completed an examination in clinical anatomy which included 40 multiple‐choice questions with five response options each. Among these, 245 (82.2%) examinees were assessed according to the number‐right scoring method (group A) while 53 (17.8%) were assessed according to the formula scoring method (group B). The prevalence of passing was significantly higher in group A than in group B, after correction of the pass and fail cutoffs for guessing (84.9% vs. 62.3%, P = 0.005), keeping a similar reliability in both groups (0.7 vs. 0.8, P = 0.094). Pearson Correlation coefficients between practical and theoretical examination scores were 0.66 [95%CI = (0.58–0.73)] and 0.72 [95%CI = (0.56–0.83)] for groups A and B, respectively. Based solely on the reliability and validity assessments, the test‐maker could therefore use either scoring rules; however, if the test‐maker also takes into account the students' ability to deduce answers with partial knowledge, then the number‐right score rule is most appropriate. Anat Sci Educ 8: 242–248. © 2014 American Association of Anatomists.