Image-Based Force Estimation in Medical Applications: A Review

• Published in: IEEE sensors journal 2021-04, Vol.21 (7), p.8805-8830
• Main Authors: Nazari, Ali A., Janabi-Sharifi, Farrokh, Zareinia, Kourosh
• Format: Article
• Language: English
• Subjects: Algorithms; Biological tissues; Computational modeling; Computer vision; Data processing; Deformable models; Deformable object; Force; learning-based force estimation; Machine learning; Medical research; Micromanipulation; minimally invasive robotic surgery; Robot sensing systems; Robotics; Robots; Sensors; soft tissue; Soft tissues; Vision systems; vision-based force estimation
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2021.3052755

Minimally invasive robotic interventions have highlighted the need to develop efficient techniques to measure forces applied to the soft tissues. Since the last decade, many scholars have focused on micro-scale and macro-scale robotic manipulations. Early articles used the model of soft tissue mathematically and tracked the displacement of the contour of the object in the vision system to provide the corresponding force to the user. Lack of knowledge of different materials and the computational complexity led to a transition from model-based to learning-based approaches to interpret the relation between object deformations, extracted from the vision system, and the real forces applied to the object. The dramatic growth of machine learning techniques and its integration with computer vision has brought novel learning-based visual data processing methods to the area. The application of the image-based force estimation methods in a controlled medical intervention has also received significant attention in the last five years. A decent number of surveys have been published on micromanipulation in recent years, especially for cell microinjection. However, the state of the art in meso- and macro-scale medical robotic interventions has not been reviewed. The aim and contribution of this paper are to fill the stated gap by reviewing the recent advances in image-based force estimation in robotic interventions. The survey shows that learning-based force estimation methods are growing significantly by using deep learning-based methods. The survey will encourage researchers and surgeons to apply learning-based algorithms to real-time medical and health-related operations.