Localization and Control of Magnetic Suture Needles in Cluttered Surgical Site with Blood and Tissue

Real-time visual localization of needles is necessary for various surgical applications, including surgical automation and visual feedback. In this study we investigate localization and autonomous robotic control of needles in the context of our magneto-suturing system. Our system holds the potentia...

Full description

Saved in:
Bibliographic Details
Main Authors: Pryor, Will, Barnoy, Yotam, Raval, Suraj, Liu, Xiaolong, Mair, Lamar, Lerner, Daniel, Erin, Onder, Hager, Gregory D., Diaz-Mercado, Yancy, Krieger, Axel
Format: Conference Proceeding
Language:English
Subjects:
Autonomous Control
Computer Vision for Medical Robotics
Image Segmentation
Location awareness
Magnetic Manipulation
Needle Localization
Needles
Reliability
Solid modeling
Surgical Robotics
Suture Needle
Three-dimensional displays
Visualization
Wireless communication
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Real-time visual localization of needles is necessary for various surgical applications, including surgical automation and visual feedback. In this study we investigate localization and autonomous robotic control of needles in the context of our magneto-suturing system. Our system holds the potential for surgical manipulation with the benefit of minimal invasiveness and reduced patient side effects. However, the nonlinear magnetic fields produce unintuitive forces and demand delicate position-based control that exceeds the capabilities of direct human manipulation. This makes automatic needle localization a necessity. Our localization method combines neural network-based segmentation and classical techniques, and we are able to consistently locate our needle with 0.73mm RMS error in clean environments and 2.72mm RMS error in challenging environments with blood and occlusion. The average localization RMS error is 2.16 mm for all environments we used in the experiments. We combine this localization method with our closed-loop feedback control system to demonstrate the further applicability of localization to autonomous control. Our needle is able to follow a running suture path in (1) no blood, no tissue; (2) heavy blood, no tissue; (3) no blood, with tissue; and (4) heavy blood, with tissue environments. The tip position tracking error ranges from 2.6mm to 3.7mm RMS, opening the door towards autonomous suturing tasks.
ISSN:2153-0858
2153-0866
DOI:10.1109/IROS51168.2021.9636441