Teaching a Robot to Grasp Real Fish by Imitation Learning from a Human Supervisor in Virtual Reality

We teach a real robot to grasp real fish, by training a virtual robot exclusively in virtual reality. Our approach implements robot imitation learning from a human supervisor in virtual reality. A deep 3D convolutional neural network computes grasps from a 3D occupancy grid obtained from depth imagi...

Full description

Saved in:
Bibliographic Details
Main Authors: Dyrstad, Jonatan S., Ruud Oye, Elling, Stahl, Annette, Reidar Mathiassen, John
Format: Conference Proceeding
Language:English
Subjects:
Cameras
Grasping
Grippers
Robots
Task analysis
Three-dimensional displays
Virtual reality
Citations: Items that cite this one
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We teach a real robot to grasp real fish, by training a virtual robot exclusively in virtual reality. Our approach implements robot imitation learning from a human supervisor in virtual reality. A deep 3D convolutional neural network computes grasps from a 3D occupancy grid obtained from depth imaging at multiple viewpoints. In virtual reality, a human supervisor can easily and intuitively demonstrate examples of how to grasp an object, such as a fish. From a few dozen of these demonstrations, we use domain randomization to generate a large synthetic training data set consisting of 100 000 example grasps of fish. Using this data set for training purposes, the network is able to guide a real robot and gripper to grasp real fish with good success rates. The newly proposed domain randomization approach constitutes the first step in how to efficiently perform robot imitation learning from a human supervisor in virtual reality in a way that transfers well to the real world.
ISSN:2153-0866
DOI:10.1109/IROS.2018.8593954