A Digital Twin Framework for Telesurgery in the Presence of Varying Network Quality of Service

• Main Authors: Bonne, Sophea, Panitch, Will, Dharmarajan, Karthik, Srinivas, Kishore, Kincade, Jerri-Lynn, Low, Thomas, Knoth, Bruce, Cowan, Cregg, Fer, Danyal, Thananjeyan, Brijen, Kerr, Justin, Ichnowski, Jeffrey, Goldberg, Ken
• Format: Conference Proceeding
• Language: English
• Subjects: Delays; Digital twins; Minimally invasive surgery; Quality of service; Teleoperators; Three-dimensional displays; Training
• ISSN: 2161-8089
• DOI: 10.1109/CASE49997.2022.9926585

Remote telesurgery can enable expert surgeons to operate on patients in distant or underserved locales. However, network instability and delays hamper long-distance communication. To address this, we explore how a "digital twin," a 3D simulator that actively mirrors a real environment, can be applied to telesurgery. We focus on the Fundamentals of Laparoscopic Surgery peg transfer surgical training task. We present a framework that enables a teleoperator to perform this task over unstable or low-bandwidth communication channels using a digital twin. The surgeon remotely teleoperates the robot in our simulator, which abstracts their motions into commands and transmits them to the real robot for semi-autonomous execution. The system executes the transfer and then sends the real state of the pegboard back to the simulator. We present experiments that demonstrate that the operation of each portion of the framework in isolation maintains a high task success rate, and that the success rate of the digital twin framework is robust to network transmission instability and delays.