Surgical Skill Can be Objectively Measured From Fluoroscopic Images Using a Novel Image-based Decision Error Analysis (IDEA) Score

• Published in: Clinical orthopaedics and related research 2021-06, Vol.479 (6), p.1386-1394
• Main Authors: Long, Steven, Thomas, Geb W., Karam, Matthew D., Marsh, J. Lawrence, Anderson, Donald D.
• Format: Article
• Language: English
• Subjects: Adult; Basic Research; Bone Wires; Clinical Competence - statistics & numerical data; Clinical trials; Decision Support Techniques; Drilling; Error analysis; Female; Femur Head - surgery; Fluoroscopy; Fractures; Hip; Hip Fractures - surgery; Hip joint; Humans; Image processing; Internship and Residency; Male; Medical Errors - statistics & numerical data; Middle Aged; Operating Rooms; Orthopedic Procedures - education; Orthopedic Procedures - methods; Orthopedics; Simulation Training; Surgeons
• ISSN: 0009-921X; 1528-1132
• DOI: 10.1097/CORR.0000000000001623

To advance orthopaedic surgical skills training and assessment, more rigorous and objective performance measures are needed. In hip fracture repair, the tip-apex distance is a commonly used summative performance metric with clear clinical relevance, but it does not capture the skill exercised during the process of achieving the final implant position. This study introduces and evaluates a novel Image-based Decision Error Analysis (IDEA) score that better captures performance during fluoroscopically-assisted wire navigation. (1) Can wire navigation skill be objectively measured from a sequence of fluoroscopic images? (2) Are skill behaviors observed in a simulated environment also exhibited in the operating room? Additionally, we sought to define an objective skill metric that demonstrates improvement associated with accumulated surgical experience. Performance was evaluated both on a hip fracture wire navigation simulator and in the operating room during actual fracture surgery. After examining fluoroscopic image sequences from 176 consecutive simulator trials (performed by 58 first-year orthopaedic residents) and 21 consecutive surgical procedures (performed by 19 different orthopaedic residents and one attending orthopaedic surgeon), three main categories of erroneous skill behavior were identified: off-target wire adjustments, out-of-plane wire adjustments, and off-target drilling. Skill behaviors were measured by comparing wire adjustments made between consecutive images against the goal of targeting the apex of the femoral head as part of our new IDEA scoring methodology. Decision error metrics (frequency, magnitude) were correlated with other measures (image count and tip-apex distance) to characterize factors related to surgical performance on both the simulator and in the operating room. An IDEA composite score integrating decision errors (off-target wire adjustments, out-of-plane wire adjustments, and off-target drilling) and the final tip-apex distance to produce a single metric of overall performance was created and compared with the number of hip wire navigation cases previously completed (such as surgeon experience levels). The IDEA methodology objectively analyzed 37,000 images from the simulator and 688 images from the operating room. The number of decision errors (7 ± 5 in the operating room and 4 ± 3 on the simulator) correlated with fluoroscopic image count (33 ± 14 in the operating room and 20 ± 11 on the simulator) in both the simulator