ARei: Augmented-Reality-Assisted Touchless Teleoperated Robot for Endoluminal Intervention

Current robotic endoluminal interventions require surgeons to hold a proximal joystick to control the distal flexible robot under 2-D X-ray guidance. However, the 2-D X-ray image is not intuitive, which not only increases the risk of surgical misoperation but also the workload of surgeons. Moreover,...

Full description

Saved in:
Bibliographic Details
Published in:IEEE/ASME transactions on mechatronics 2022-10, Vol.27 (5), p.3144-3154
Main Authors: Lin, Zecai, Gao, Anzhu, Ai, Xiaojie, Gao, Hongyan, Fu, Yili, Chen, Weidong, Yang, Guang-Zhong
Format: Article
Language:English
Subjects:
Augmented reality
Augmented reality (AR)
Calibration
continuum robot
Error analysis
flexible instrument
Gesture recognition
Helmet mounted displays
Instruments
Resists
Robot control
Robot sensing systems
robotic surgery
Robots
Sensors
Shape
shape sensing
Surgeons
Three-dimensional displays
touchless teleoperation
Tracking systems
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Current robotic endoluminal interventions require surgeons to hold a proximal joystick to control the distal flexible robot under 2-D X-ray guidance. However, the 2-D X-ray image is not intuitive, which not only increases the risk of surgical misoperation but also the workload of surgeons. Moreover, contact teleoperation exposes surgeons to the potentially infectious environment. To address it, this article proposes an augmented-reality-assisted touchless teleoperated robot for endoluminal intervention, called ARei. It aims to provide immersive experiences with augmented information. The robot integrates perceptual information obtained from an electromagnetic (EM) sensor, a shape sensor, and the virtual anatomy into a head-mounted display (HMD). The touchless teleoperation is used to control the robot, with the assistance of gesture recognition technology. Results show that the mean error of the calibration between the HMD and the EM tracking system is 4.67 mm, and the mean distance error between the points measured by the EM sensor and the points obtained by shape reconstruction with calibration is 5.19 mm (3.05%).
ISSN:1083-4435
1941-014X
DOI:10.1109/TMECH.2021.3105536