Endobronchial Ultrasound Skills and Tasks Assessment Tool: Assessing the Validity Evidence for a Test of Endobronchial Ultrasound-guided Transbronchial Needle Aspiration Operator Skill

• Published in: American journal of respiratory and critical care medicine 2012-10, Vol.186 (8), p.773-779
• Main Authors: DAVOUDI, Mohsen, COLT, Henri G, OSANN, Kathryn E, LAMB, Carla R, MULLON, John J
• Format: Article
• Language: English
• Subjects: Airway management; Anesthesia. Intensive care medicine. Transfusions. Cell therapy and gene therapy; Benchmarking; Biological and medical sciences; Bronchoscopy; Bronchoscopy - education; Clinical Competence; Clinical death. Palliative care. Organ gift and preservation; Coronary vessels; Education; Educational Measurement; Endoscopic Ultrasound-Guided Fine Needle Aspiration; Humans; Intensive care medicine; Knowledge acquisition; Medical sciences; Multivariate Analysis; Observer Variation; Pulmonary arteries; Questionnaires; Regression Analysis; Reproducibility of Results; Self evaluation; Self-Assessment; Simulation; Skills; Tomography; Ultrasonic imaging; United States; Validity
• ISSN: 1073-449X; 1535-4970
• DOI: 10.1164/rccm.201111-1968OC

Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is becoming standard of care for the sampling of mediastinal adenopathy. The need for a safe, effective, accurate procedure makes EBUS-TBNA ideal for mastery training and testing. The Endobronchial Ultrasound Skills and Tasks Assessment Tool (EBUS-STAT) was created as an objective competency-oriented assessment tool of EBUS-TBNA skills and knowledge. This study demonstrates the reliability and validity evidence of this tool. The EBUS-STAT objectively scores the EBUS-TBNA operator's skills, including atraumatic airway introduction and navigation, ultrasound image acquisition and optimization, identification of mediastinal nodal and vascular structures, EBUS-TBNA sampling, and recognition of EBUS/computed tomography images of mediastinal structures. It can be administered at the bedside or using combination of low- and high-fidelity simulation platforms. Two independent testers administered the EBUS-STAT to 24 operators at three levels of EBUS-TBNA experience (8 beginners, 8 intermediates, and 8 experienced) at three institutions; operators were also asked to self-assess their skills. Scores were analyzed for intertester reliability, correlation with prior EBUS-TBNA experience, and association with self-assessments. Intertester reliability between testers was very high (r = 0.9991, P < 0.00005). Mean EBUS-STAT scores for beginner, intermediate, and experienced groups, respectively, were 31.1, 74.9, and 93.6 out of 100 (F(2,21) = 118.6, P < 0.0001). Groups were nonoverlapping: post hoc tests showed each group differed significantly from the others (P < 0.001). Self-assessments corresponded closely to actual EBUS-STAT scores (r(2) = 0.81, P < 0.001). The EBUS-STAT can be used to reliably and objectively score and classify EBUS-TBNA operators from novice to expert. Its use to assess and document the acquisition of knowledge and skill is a step toward the goal of mastery training in EBUS-TBNA.