Augmented reality headsets for surgical guidance: the impact of holographic model positions on user localisation accuracy

• Published in: Virtual reality : the journal of the Virtual Reality Society 2024-03, Vol.28 (2), p.68, Article 68
• Main Authors: Pérez-Pachón, Laura, Sharma, Parivrudh, Brech, Helena, Gregory, Jenny, Lowe, Terry, Poyade, Matthieu, Gröning, Flora
• Format: Article
• Language: English
• Subjects: Apexes; Artificial Intelligence; Augmented reality; Computer Graphics; Computer Science; Error analysis; Headsets; Image Processing and Computer Vision; Localization; Original Article; Outliers (statistics); Surgeons; Tumors; User Interfaces and Human Computer Interaction
• ISSN: 1434-9957; 1359-4338; 1434-9957
• DOI: 10.1007/s10055-024-00960-x

Novel augmented reality headsets such as HoloLens can be used to overlay patient-specific virtual models of resection margins on the patient’s skin, providing surgeons with information not normally available in the operating room. For this to be useful, surgeons wearing the headset must be able to localise virtual models accurately. We measured the error with which users localise virtual models at different positions and distances from their eyes. Healthy volunteers aged 20–59 years ( n  = 54) performed 81 exercises involving the localisation of a virtual hexagon’s vertices overlaid on a monitor surface. Nine predefined positions and three distances between the virtual hexagon and the users’ eyes (65, 85 and 105 cm) were set. We found that, some model positions and the shortest distance (65 cm) led to larger localisation errors than other positions and larger distances (85 and 105 cm). Positional errors of more than 5 mm and 1–5 mm margin errors were found in 29.8% and over 40% of cases, respectively. Strong outliers were also found (e.g. margin shrinkage of up to 17.4 mm in 4.3% of cases). The measured errors may result in poor outcomes of surgeries: e.g. incomplete tumour excision or inaccurate flap design, which can potentially lead to tumour recurrence and flap failure, respectively. Reducing localisation errors associated with arm reach distances between the virtual models and users’ eyes is necessary for augmented reality headsets to be suitable for surgical purposes. In addition, training surgeons on the use of these headsets may help to minimise localisation errors.