Standardized Multi-Institutional Mock Oral Examination: A Feasible and Valuable Educational Experience for General Surgery Residents

• Published in: Journal of surgical education 2020-11, Vol.77 (6), p.1568-1576
• Main Authors: Lu, Yang, Miranda, Ruth, Quach, Chi, Girgis, Mark, Lewis, Catherine E., Tillou, Areti, Chen, Formosa
• Format: Article
• Language: English
• Subjects: American Board of Surgery Certifying Exam; Clinical Competence; Educational Measurement; General Surgery - education; General surgery, mock orals; Humans; Internship and Residency; Interpersonal and communication skills; Mock oral examination; Reproducibility of Results; Surgical education; Surveys and Questionnaires
• ISSN: 1931-7204; 1878-7452
• DOI: 10.1016/j.jsurg.2020.05.015

•General Surgery residents and faculty perceived the standardized multi-institutional MOE to be a highly acceptable and valuable educational experience.•Developing a repertoire of prescripted standardized exam scenarios made it feasible to recruit faculty participants.•Standardizing test scenarios and grading rubrics created a more consistent exam experience across participants. Mock oral examinations (MOE) are used to prepare residents and assess their readiness for the American Board of Surgery Certifying Exam (ABSCE). Delivery of MOEs varies by institution and previous studies have demonstrated significant implementation barriers such as availability of faculty examiners and exam scenarios. To assess the value and participant satisfaction of a standardized multi-institutional MOE for general surgery residents. Thirty-three general surgery residents and 37 faculty members from 3 institutions participated in a regional MOE. Residents were examined in three 20-minute sessions. Faculty examiners were given a wide selection of prescripted exam scenarios and instructed to use standardized grading rubrics during a brief orientation on the day of the exam. All participants were surveyed on their overall experience. Of 33 participating residents, 26 (79%) passed the MOE (92% of R5, 91% R4, and 50% of R3). Response rates were 91% for residents, and 57% for faculty members respectively. Most respondents were satisfied with the overall exam experience (88%), standardized question quality (86%) and question variety (82%). A total of 92% of respondents agreed that the time, effort, and cost of the MOE was justified by its educational value to residents. Higher medical knowledge ratings assigned by faculty examiners correlated with stronger trainee performance (β = 0.48; 95% confidence interval [CI] 0.29-0.66), while patient care and interpersonal communication skill ratings were not associated with trainee performance. The standardized grading rubric achieved moderate inter-rater reliability among examiner pairs with 70.6% agreement (Kappa 0.47). General Surgery residents and faculty perceived the standardized multi-institutional MOE to be a highly satisfactory educational experience and valuable assessment tool. Developing a repertoire of scripted exam scenarios made it feasible to recruit sufficient faculty participants, and standardizing grading rubrics allowed for a consistent exam experience with moderate inter-rater reliability.