Video assessment of basic surgical trainees' operative skills

• Published in: The American journal of surgery 2008-08, Vol.196 (2), p.265-272
• Main Authors: Driscoll, Peter J., M.R.C.S. (Ed.), Paisley, Anna M., F.R.C.S. (Gen. Surg.), Paterson-Brown, Simon, F.R.C.S. (Ed.)
• Format: Article
• Language: English
• Subjects: Assessment; Biological and medical sciences; Clinical Competence; Educational Measurement - methods; Feasibility Studies; General aspects; General Surgery - education; Hernia, Inguinal - surgery; Humans; Likert scale; Medical sciences; Reproducibility of Results; Skills; Surgery; Surgical Training; Training; Validity; Video; Videotape Recording
• ISSN: 0002-9610; 1879-1883
• DOI: 10.1016/j.amjsurg.2007.09.044

Abstract Background We previously described a feasible, reliable, valid, and acceptable clinical assessment form for basic surgical trainees (BSTs). We now recently assessed tissue-handling skills using real-time assessment (RTA) and video assessment (VA) and addressed feasibility, reliability, validity, and trainer–trainee agreement using the same assessment form. Methods Nine BSTs were videotaped as they performed open inguinal hernia repairs at 6 and 12 months of surgical training. Edited videotapes were independently rated by 7 consultants and 5 trainees using the technical part of the Edinburgh BST Assessment Form (EBSTAF-Tech), the Toronto Global Rating Scale of Surgical Performance (Toronto scale), and a visual analogue scale. Results RTA and VA both proved to be reliable tools (RTA α ≥ .85; VA α ≥ .76; interclass correlation coefficient [ICC] ≥ .69, rho ≥ .694 [ P ≤ .004]), but RTA was not construct valid. VA scores distinguished consultant from trainee operators (Mann-Whitney P < .02), with trainers able to discriminate between trainee levels (Wilcoxon P = .01 to .023). Concurrent validity was demonstrated by the VA (trainers tau-b = .71 to .79 [ P < .001] and trainees tau-b = .79 to .82 [ P < .001]) with good trainer–trainee agreement (EBSTAF-Tech tau-b = .35 [ P = .03), Toronto tau-b = .46 [ P = .006), and visual analogue scale tau-b = .46 [P = .006]). Conclusions VA of BST tissue-handling skills is feasible, reliable, valid, and highly sensitive. It may also improve trainee self-assessment skills by promoting reflective practice.