Do United States Medical Licensing Examination (USMLE) Scores Predict In-Training Test Performance for Emergency Medicine Residents?

• Published in: The Journal of emergency medicine 2010, Vol.38 (1), p.65-69
• Main Authors: Thundiyil, Josef G., MD, MPH, Modica, Renee F., MD, MSED, Silvestri, Salvatore, MD, Papa, Linda, MD
• Format: Article
• Language: English
• Subjects: ABEM; Certification; Educational Measurement; Emergency; Emergency Medicine - education; emergency medicine training; Humans; in-training scores; Internship and Residency; Licensure, Medical; residency performance; Retrospective Studies; School Admission Criteria; United States; USMLE scores
• ISSN: 0736-4679; 2352-5029
• DOI: 10.1016/j.jemermed.2008.04.010

Abstract Background: Residency selection committees commonly utilize USMLE scores as criteria to screen residency applicants. Objectives: The purpose of this study is to evaluate the relationship between United States Medical Licensing Examination (USMLE) and American Board of Emergency Medicine (ABEM) in-training examination scores (ITEs). Methods: In an Accreditation Council for Graduate Medical Education-accredited emergency medicine residency program, data were collected for this retrospective cohort study for the classes of 2002–2006. USMLE Step 1 and 2 scores and the ABEM ITEs were recorded for each post-graduate year (PGY) within the aforementioned time frame. Step 1 and 2 scores were compared to consecutive PGY ABEM ITEs to evaluate for an association. Results: There were 51 USMLE Step 1 and 39 Step 2 scores available for comparison with 153 ABEM ITEs. The mean USMLE Step 1 and Step 2 scores were 228.9 (range 197–252) and 228.4 (range 168–259), respectively. The mean in-training percentiles for the PGY 1, 2, and 3 years were 40.4, 68.3, and 81.7, respectively. The R-squared values for the Step 1 scores compared to the PGY 1, 2, and 3 years' ITEs were 0.25, 0.18, and 0.16, respectively. The R-squared values for Step 2 scores as compared to the ABEM ITEs for the PGY 1, 2, and 3 years were 0.43, 0.44, and 0.38, respectively. Residents who scored below 200 on either USMLE Step 1 or Step 2 had significantly lower mean ABEM ITEs than residents who scored above 200 ( p < 0.05) and were 10-fold more likely than residents who scored above 220 to score below the 70th percentile in their PGY3 ABEM ITE. Conclusions: USMLE Step 1 scores are mildly correlated and Step 2 scores are moderately correlated with ABEM ITEs. Scoring below 200 on either test is associated with significantly lower ABEM ITEs.