Augmented Reality to Improve Surgical Simulation: Lessons Learned Towards the Design of a Hybrid Laparoscopic Simulator for Cholecystectomy

• Published in: IEEE transactions on biomedical engineering 2019-07, Vol.66 (7), p.2091-2104
• Main Authors: Viglialoro, Rosanna M., Esposito, Nicola, Condino, Sara, Cutolo, Fabrizio, Guadagni, Simone, Gesi, Marco, Ferrari, Mauro, Ferrari, Vincenzo
• Format: Article
• Language: English
• Subjects: Augmented reality; Blood vessels; Cholecystectomy; cholecystectomy training; Computer applications; Electromagnetic fields; hybrid simulators; Initial specifications; Laparoscopes; laparoscopic simulation; Laparoscopy; Minimally invasive surgery; Modularity; Nerves; physical anatomical model; Real time; Realism; Robustness; Simulation; Simulators; Solid modeling; Spare parts; Specifications; Surgical simulator; Task analysis; Tracking; Training; Virtual reality; Visualization
• ISSN: 0018-9294; 1558-2531
• DOI: 10.1109/TBME.2018.2883816

Hybrid surgical simulators based on augmented reality (AR) solutions benefit from the advantages of both the box trainers and the virtual reality simulators. This paper reports on the results of a long development stage of a hybrid simulator for laparoscopic cholecystectomy that integrates real and the virtual components. We first outline the specifications of the AR simulator and then we explain the strategy adopted for implementing it based on a careful selection of its simulated anatomical components, and characterized by a real-time tracking of both a target anatomy and of the laparoscope. The former is tracked by means of an electromagnetic field generator, while the latter requires an additional camera for video tracking. The new system was evaluated in terms of AR visualization accuracy, realism, and hardware robustness. Obtained results show that the accuracy of AR visualization is adequate for training purposes. The qualitative evaluation confirms the robustness and the realism of the simulator. In conclusion, the proposed AR simulator satisfies all the initial specifications in terms of anatomical appearance, modularity, reusability, minimization of spare parts cost, and ability to record surgical errors and to track in real-time the Calot's triangle and the laparoscope. Thus, the proposed system could be an effective training tool for learning the task of identification and isolation of Calot's triangle in laparoscopic cholecystectomy. Moreover, the presented strategy could be applied to simulate other surgical procedures involving the task of identification and isolation of generic tubular structures, such as blood vessels, biliary tree, and nerves, which are not directly visible.