Laparoscopy with augmented reality adaptations

• Published in: Journal of biomedical informatics 2020-07, Vol.107, p.103463-103463, Article 103463
• Main Authors: Zorzal, Ezequiel Roberto, Campos Gomes, José Miguel, Sousa, Maurício, Belchior, Pedro, da Silva, Pedro Garcia, Figueiredo, Nuno, Lopes, Daniel Simões, Jorge, Joaquim
• Format: Article
• Language: English
• Subjects: Augmented reality; Hand-eye coordination; Laparoscopic surgery; Multimodal interface; Surgery
• ISSN: 1532-0464; 1532-0480
• DOI: 10.1016/j.jbi.2020.103463

[Display omitted] •A solution to visualise the laparoscopic video while not forcing the surgeon to assume unnecessarily uncomfortable positions.•A completely hands-free approach, using both head gaze and foot movement as sources of input.•Improvements to surgeon-to-surgeon communication using the head gaze as a means of pointing. One of the most promising applications of Optical See-Through Augmented Reality is minimally laparoscopic surgery, which currently suffers from problems such as surgeon discomfort and fatigue caused by looking at a display positioned outside the surgeon’s visual field, made worse by the length of the procedure. This fatigue is especially felt on the surgeon’s neck, as it is strained from adopting unnatural postures in order to visualise the laparoscopic video feed. Throughout this paper, we will present work in Augmented Reality, as well as developments in surgery and Augmented Reality applied to both surgery in general and laparoscopy in particular to address these issues. We applied user and task analysis methods to learn about practices performed in the operating room by observing surgeons in their working environment in order to understand, in detail, how they performed their tasks and achieved their intended goals. Drawing on observations and analysis of video recordings of laparoscopic surgeries, we identified relevant constraints and design requirements. Besides proposals to approach the ergonomic issues, we present a design and implementation of a multimodal interface to enhance the laparoscopic procedure. Our method makes it more comfortable for surgeons by allowing them to keep the laparoscopic video in their viewing area regardless of neck posture. Also, our interface makes it possible to access patient imaging data without interrupting the operation. It also makes it possible to communicate with team members through a pointing reticle. We evaluated how surgeons perceived the implemented prototype, in terms of usefulness and usability, via a think-aloud protocol to conduct qualitative evaluation sessions which we describe in detail in this paper. In addition to checking the advantages of the prototype as compared to traditional laparoscopic settings, we also conducted a System Usability Scale questionnaire for measuring its usability, and a NASA Task Load Index questionnaire to rate perceived workload and to assess the prototype effectiveness. Our results show that surgeons consider that our prototype can improve surgeo