Evaluating the Effectiveness of a Mock Oral Educational Program

• Published in: The Journal of surgical research 2016-10, Vol.205 (2), p.305-311
• Main Authors: Fischer, Laura E., MD, MS, Snyder, Mara, MA, Sullivan, Sarah A., PhD, Foley, Eugene F., MD, Greenberg, Jacob A., MD, EdM
• Format: Article
• Language: English
• Subjects: ABSITE; American Board of surgery; Certification - statistics & numerical data; Certifying exam; Clinical Competence - statistics & numerical data; Educational Measurement - methods; General Surgery - education; Humans; Internship and Residency; Mock oral exam; Program Evaluation; Qualifying exam; Resident education; Retrospective Studies; Surgery; Wisconsin
• ISSN: 0022-4804; 1095-8673
• DOI: 10.1016/j.jss.2016.06.088

Abstract Background In order to obtain board certification, the American Board of Surgery requires graduates of general surgery training programs to pass both the written Qualifying Exam (QE) and the oral Certifying Exam (CE). In 2015, the pass rates for the QE and CE were 80% and 77%, respectively. In the 2011-2012 academic year, the University of Wisconsin instituted a mandatory, faculty-led, monthly CE preparation educational program (CE Prep) as a supplement to their existing annual mock oral exam. We hypothesized that the implementation of these sessions would improve the first time pass rate for residents taking the ABS CE at our institution. Secondary outcomes studied were QE pass rate, correlation with ABSITE and mock oral exam scores, cost and type of study materials used, perception of exam difficulty, and applicant preparedness. Methods A sixteen question survey was sent to 57 out of 59 residents who attended the University of Wisconsin between the years of 2007 and 2015. Email addresses for two former residents could not be located. De-identified data for the ABSITE and first time pass rates for the QE and CE exam were retrospectively collected and analyzed along with survey results. Statistical analysis was performed using SPSS version 22 (IBM Corp., Armonk, NY). P-values less than 0.05 were considered significant. Results Survey response rate was 77.2%. Of the residents who have attempted the CE, first time pass rate was 76.0% (19 of 25) prior to the implementation of the formal CE Prep and 100% (22 of 22) after (p = 0.025). Absolute ABSITE score, and mock oral annual exam grades were significantly improved after the CE Prep was initiated (p-values < 0.001 and 0.003, respectively), however, ABSITE percentile was not significantly different (p = 0.415). ABSITE raw score and percentile, as well as mock oral annual exam scores were significantly associated with passing the QE (0.032, 0.027, and 0.020 respectively), while mock oral annual exam scores alone were associated with passing the CE (p = 0.001). Survey results showed that residents perceived the CE to be easier than the annual mock oral after the institution of the CE Prep course (p = 0.036), however, there was no difference in their perception of preparedness. Overall, applicants felt extremely prepared for the CE (4.70 ± 0.5, Likert scale 1-5). Conclusions Formal educational programs instituted during residency can improve resident performance on the ABS certifying exam. The institutio