Sim-To-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery

Automation holds the potential to assist surgeons in robotic interventions, shifting their mental work load from visuomotor control to high level decision making. Reinforcement learning has shown promising results in learning complex visuomotor policies, especially in simulation environments where m...

Full description

Saved in:
Bibliographic Details
Published in:IEEE robotics and automation letters 2023-02, Vol.8 (2), p.1-8
Main Authors: Scheikl, Paul Maria, Tagliabue, Eleonora, Gyenes, Balazs, Wagner, Martin, Dall'Alba, Diego, Fiorini, Paolo, Mathis-Ullrich, Franziska
Format: Article
Language:English
Subjects:
Color imagery
Computer Vision for Medical Robotics
Decision making
Deformation effects
Domains
Formability
Grippers
Image manipulation
Medical imaging
Medical robotics
Mental work
Policies
Reinforcement Learning
Robotic surgery
Robotics
Robots
Simulation
Surgery
Surgical Robotics: Laparoscopy
Task analysis
Training
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automation holds the potential to assist surgeons in robotic interventions, shifting their mental work load from visuomotor control to high level decision making. Reinforcement learning has shown promising results in learning complex visuomotor policies, especially in simulation environments where many samples can be collected at low cost. A core challenge is learning policies in simulation that can be deployed in the real world, thereby overcoming the sim-to-real gap. In this work, we bridge the visual sim-to-real gap with an image-based reinforcement learning pipeline based on pixel-level domain adaptation and demonstrate its effectiveness on an image-based task in deformable object manipulation. We choose a tissue retraction task because of its importance in clinical reality of precise cancer surgery. After training in simulation on domain-translated images, our policy requires no retraining to perform tissue retraction with a 50\% success rate on the real robotic system using raw RGB images. Furthermore, our sim-to-real transfer method makes no assumptions on the task itself and requires no paired images. This work introduces the first successful application of visual sim-to-real transfer for robotic manipulation of deformable objects in the surgical field, which represents a notable step towards the clinical translation of cognitive surgical robotics.
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2022.3227873