A Generative Model-Based Predictive Display for Robotic Teleoperation

We propose a new generative model-based predictive display for robotic teleoperation over high-latency communication links. Our method is capable of rendering photo-realistic images of the scene to the human operator in real time from RGB-D images acquired by the remote robot. A preliminary explorat...

Full description

Saved in:
Bibliographic Details
Main Authors: Xie, Bowen, Han, Mingjie, Jin, Jun, Barczyk, Martin, Jagersand, Martin
Format: Conference Proceeding
Language:English
Subjects:
Pipelines
Predictive models
Semantics
Solid modeling
Three-dimensional displays
Two dimensional displays
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a new generative model-based predictive display for robotic teleoperation over high-latency communication links. Our method is capable of rendering photo-realistic images of the scene to the human operator in real time from RGB-D images acquired by the remote robot. A preliminary exploration stage is used to build a coarse 3D map of the remote environment and to train a generative model, both of which are then used to generate photo-realistic images for the human operator based on the commanded pose of the robot. Data captured by the remote robot is used to dynamically update the 3D map, enabling teleoperation in the presence of new and relocated objects. Various experiments validate our proposed method's performance and benefits over alternative methods.
ISSN:2577-087X
DOI:10.1109/ICRA48506.2021.9561787