Affordance Learning from Play for Sample-Efficient Policy Learning

Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal. To this end, we propose a novel approach that extracts a self-supervised visu...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2022-03
Main Authors: Borja-Diaz, Jessica, Mees, Oier, Kalweit, Gabriel, Hermann, Lukas, Boedecker, Joschka, Burgard, Wolfram
Format: Article
Language:English
Subjects:
Algorithms
Deep learning
Machine learning
Motion planning
Optimization
Policies
Robots
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal. To this end, we propose a novel approach that extracts a self-supervised visual affordance model from human teleoperated play data and leverages it to enable efficient policy learning and motion planning. We combine model-based planning with model-free deep reinforcement learning (RL) to learn policies that favor the same object regions favored by people, while requiring minimal robot interactions with the environment. We evaluate our algorithm, Visual Affordance-guided Policy Optimization (VAPO), with both diverse simulation manipulation tasks and real world robot tidy-up experiments to demonstrate the effectiveness of our affordance-guided policies. We find that our policies train 4x faster than the baselines and generalize better to novel objects because our visual affordance model can anticipate their affordance regions.
ISSN:2331-8422