Multiphysics simulation and design framework for developing a vision-based tactile sensor with force estimation and slip detection capabilities

• Published in: Sensors and actuators. A. Physical. 2024-11, Vol.378, p.115761, Article 115761
• Main Authors: Mirzaee, Mohammad Amin, Sadighi, Ali
• Format: Article
• Language: English
• Subjects: Force sensor; Machine learning; Optical simulation; Tactile sensing; Uncertainty analysis; Vision-based tactile sensing
• ISSN: 0924-4247
• DOI: 10.1016/j.sna.2024.115761

Recent achievements in the field of tactile sensing has enhanced the dexterity of robots interacting with their environments. Superior to other transduction mechanisms, vision-based tactile sensing enables high-resolution sensing of the contact surface. Accordingly, many vision-based tactile sensors have been introduced in recent years, all of which are designed and fabricated based on a trial-and-error approach with limited prior system modeling and simulation. In this study, we introduce a detailed framework for the development of a vision-based tactile sensor with markers motion mechanism. We start with the design and coupled simulation of the mechanical and optical subsystems followed by fabrication, testing, and validation of the proposed sensor. An uncertainty analysis is carried out during the design stage where various errors are propagated to determine the expected sensor accuracy. The simulated sensor showed a maximum error of 0.15 newtons with the error for the camera distance to the rigid membrane having the highest contribution. After fabrication of the sensor, it is observed that there is a very good agreement between simulation and experimental results. The sensor was tested for sensing tangential forces (up to slip) along two directions and normal force (up to 5 newtons) at 30 Hz rate with measurement errors (95 % confidence interval) of 0.39 newton, 0.11 newton, and 0.12 newton, respectively. The sensor detected the slip in 97 % of the tests with an average latency of 10 frames. [Display omitted] •Multiphysics simulation framework to assess a vision-based tactile sensor.•Uncertainty analysis in simulation stage to obtain underlying errors.•Sensor design and fabrication and comparing experimental and simulation results.•Force estimation using machine-learning methods and comparing the results.•Slip detection using estimated force variance with 97 % accuracy in 10 frames.