eAssessment: development of an electronic version of the Objective Structured Assessment of Debriefing tool to streamline evaluation of video recorded debriefings

• Published in: Journal of the American Medical Informatics Association : JAMIA 2018-10, Vol.25 (10), p.1284-1291
• Main Authors: Zamjahn, John B, Baroni de Carvalho, Raquel, Bronson, Megan H, Garbee, Deborah D, Paige, John T
• Format: Article
• Language: English
• Subjects: Clinical Competence; Education, Medical; Educational Measurement - methods; Feedback; High Fidelity Simulation Training; Humans; Research and Applications; User-Computer Interface
• ISSN: 1067-5027; 1527-974X
• DOI: 10.1093/jamia/ocy113

The Objective Structured Assessment of Debriefing (OSAD) is an evidence-based, 8-item tool that uses a behaviorally anchored rating scale in paper-based form to evaluate the quality of debriefing in medical education. The objective of this project was twofold: 1) to create an easy-to-use electronic format of the OSAD (eOSAD) in order to streamline data entry; and 2) to pilot its use on videoed debriefings. The eOSAD was developed in collaboration with the LSU Health New Orleans Epidemiology Data Center using SurveyGizmo (Widgix Software, LLC, Boulder, CO, USA) software. The eOSAD was then piloted by 2 trained evaluators who rated 37 videos of faculty teams conducting pre-briefing and debriefing after a high-fidelity trauma simulation. Inter-rater reliability was assessed, and evaluators' qualitative feedback was obtained. Inter-rater reliability was good [prebrief, intraclass correlation coefficient, ICC = 0.955 (95% CI, 0.912-0.977), P