Grasp State Assessment of Deformable Objects Using Visual-Tactile Fusion Perception

Humans can quickly determine the force required to grasp a deformable object to prevent its sliding or excessive deformation through vision and touch, which is still a challenging task for robots. To address this issue, we propose a novel 3D convolution-based visual-tactile fusion deep neural networ...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2020-06
Main Authors: Cui, Shaowei, Wang, Rui, Wei, Junhang, Li, Fanrong, Wang, Shuo
Format: Article
Language:English
Subjects:
Adaptive control
Artificial neural networks
Automatic control
Convolution
Deformation
Formability
Grasping (robotics)
Machine learning
Robot arms
Sliding
Tactile sensors (robotics)
Touch
Wrist
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Humans can quickly determine the force required to grasp a deformable object to prevent its sliding or excessive deformation through vision and touch, which is still a challenging task for robots. To address this issue, we propose a novel 3D convolution-based visual-tactile fusion deep neural network (C3D-VTFN) to evaluate the grasp state of various deformable objects in this paper. Specifically, we divide the grasp states of deformable objects into three categories of sliding, appropriate and excessive. Also, a dataset for training and testing the proposed network is built by extensive grasping and lifting experiments with different widths and forces on 16 various deformable objects with a robotic arm equipped with a wrist camera and a tactile sensor. As a result, a classification accuracy as high as 99.97% is achieved. Furthermore, some delicate grasp experiments based on the proposed network are implemented in this paper. The experimental results demonstrate that the C3D-VTFN is accurate and efficient enough for grasp state assessment, which can be widely applied to automatic force control, adaptive grasping, and other visual-tactile spatiotemporal sequence learning problems.
ISSN:2331-8422