The National Emergency Medicine Fourth-year Student (M4) Examinations: Updates and Performance

• Published in: The Journal of emergency medicine 2016-01, Vol.50 (1), p.128-134
• Main Authors: Heitz, Corey R., MD, Lawson, Luan, MD, Beeson, Michael, MD, MBA, Miller, Emily S., MD
• Format: Article
• Language: English
• Subjects: assessment; CDEM; Clinical Clerkship; Clinical Competence; Education, Medical, Undergraduate; Educational Measurement - methods; Emergency; Emergency Medicine - education; examination; Humans; medical student
• ISSN: 0736-4679; 2352-5029
• DOI: 10.1016/j.jemermed.2015.06.072

Abstract Background Version 1 (V1) of the National Emergency Medicine Fourth-year Student (EM M4) Examination was released in 2011 and revised along with release of V2 in 2012. Each examination contains 50 multiple-choice questions designed to assess knowledge in the EM M4 clerkship curriculum. Development and initial performance data were described previously. Objective To provide updated V1 performance data, describe development and revision of V2, and to compare performance between academic years and examination forms, and within academic years. Methods Examinations are administered at www.saemtests.org with ongoing performance data provided. After 1 year of use, nine questions on V2 were revised, five because of low discriminatory ability and four because of excessive difficulty. Revision or replacement was done in accordance with the National Board of Medical Examiners (NBME) Item Writing Guidelines. Mean scores were compared for V1 between academic years (i.e., July 2011−June 2012 vs. July 2012−June 2013), V2 compared with V1, and for each examination version for early and late test takers. Results V1 has been administered >10,000 times since its release, and the current form mean is 81.5% (SD 3.7). Average discriminatory value (rpb) is 0.204. V2 has been administered >1500 times, with a mean score of 78.4% (SD 4.4) and average rpb 0.253. V1 and V2 current means differ statistically. Scores from examinees completing V1 or V2 early vs. late in the academic year differ statistically. Conclusions Performance data for V1 remain stable after 2 years. Revisions of poorly performing questions improved question performance on V2. Questions with low rpb or low pdiff will continue to be revised annually. While examination forms differ statistically, the practical utility of the differences is not defined.