Keypoint Localization Based on Convolutional Neural Network for Robotic Implantation of Flexible Micro-Electrodes

Visual localization of micro flexible electrode and implant needle is an important task for robotic flexible electrode implantation. Magnification switch, occlusion, defocus, illumination changes in microscopic imaging produce challenges for this task. We propose the Keypoint Localization and Angle...

Full description

Saved in:
Bibliographic Details
Main Authors: Liang, Wenliang, Qin, Fangbo, Han, Xinyong, Zhang, Dapeng
Format: Conference Proceeding
Language:English
Subjects:
Electrodes
Jacobian matrices
Lighting
Location awareness
Robot kinematics
Training
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Visual localization of micro flexible electrode and implant needle is an important task for robotic flexible electrode implantation. Magnification switch, occlusion, defocus, illumination changes in microscopic imaging produce challenges for this task. We propose the Keypoint Localization and Angle Estimation Network (KLAE-Net) based on convolutional neural networks. KLAE-Net has two branches: the keypoint localization branch for obtaining the coordinates of electrode and needle in image space; the angle estimation branch for monitoring the inclination of needle. Attention mechanism and deformable convolution are used to improve the model's performance. For training and evaluation under the flexible electrode implantation task, we construct a novel dataset containing 1000 images covering various conditions. An image Jacobian matrix based alignment control method is designed, to realize the robotic alignment between needle and electrode. A series of experiments are conducted with the dataset and an implantation robot system.
ISSN:2161-8089
DOI:10.1109/CASE49997.2022.9926601