Robotic Manipulation in Dynamic Scenarios via Bounding-Box-Based Hindsight Goal Generation

By relabeling past experience with heuristic or curriculum goals, state-of-the-art reinforcement learning (RL) algorithms such as hindsight experience replay (HER), hindsight goal generation (HGG), and graph-based HGG (G-HGG) have been able to solve challenging robotic manipulation tasks in multigoa...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2023-08, Vol.34 (8), p.5037-5050
Main Authors: Bing, Zhenshan, Alvarez, Erick, Cheng, Long, Morin, Fabrice O., Li, Rui, Su, Xiaojie, Huang, Kai, Knoll, Alois
Format: Article
Language:English
Subjects:
Algorithms
Barriers
Curricula
Euclidean geometry
Graph representations
Graphical representations
Heuristic algorithms
Hindsight experience replay (HER)
Image reconstruction
Machine learning
Manipulator dynamics
path planning
Reinforcement learning
reinforcement learning (RL)
robotic arm manipulation
Robots
Task analysis
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:By relabeling past experience with heuristic or curriculum goals, state-of-the-art reinforcement learning (RL) algorithms such as hindsight experience replay (HER), hindsight goal generation (HGG), and graph-based HGG (G-HGG) have been able to solve challenging robotic manipulation tasks in multigoal settings with sparse rewards. HGG outperforms HER in challenging tasks in which goals are difficult to explore by learning from a curriculum, in which intermediate goals are selected based on the Euclidean distance to target goals. G-HGG enhances HGG by selecting intermediate goals from a precomputed graph representation of the environment, which enables its applicability in an environment with stationary obstacles. However, G-HGG is not applicable to manipulation tasks with dynamic obstacles, since its graph representation is only valid in static scenarios and fails to provide any correct information to guide the exploration. In this article, we propose bounding-box-based HGG (Bbox-HGG), an extension of G-HGG selecting hindsight goals with the help of image observations of the environment, which makes it applicable to tasks with dynamic obstacles. We evaluate Bbox-HGG on four challenging manipulation tasks, where significant enhancements in both sample efficiency and overall success rate are shown over state-of-the-art algorithms. The videos can be viewed at https://videoviewsite.wixsite.com/bbhgg .
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2021.3124366