Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination

Meta reinforcement learning (Meta RL) has been amply explored to quickly learn an unseen task by transferring previously learned knowledge from similar tasks. However, most state-of-the-art Meta RL algorithms require the meta-training tasks to have a dense coverage of the task distribution and a gre...

Full description

Saved in:
Bibliographic Details
Published in:IEEE robotics and automation letters 2024-11, Vol.9 (11), p.9701-9708
Main Authors: Wen, Lu, Tseng, Eric H., Peng, Huei, Zhang, Songan
Format: Article
Language:English
Subjects:
Algorithms
Context
Decoding
deep learning methods
Interpolation
Linear programming
machine learning for robot control
Metalearning
Reinforcement learning
Task analysis
Training
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Meta reinforcement learning (Meta RL) has been amply explored to quickly learn an unseen task by transferring previously learned knowledge from similar tasks. However, most state-of-the-art Meta RL algorithms require the meta-training tasks to have a dense coverage of the task distribution and a great amount of data for each of them. In this letter, we propose MetaDreamer, a context-based Meta RL algorithm that requires less real training tasks and data by doing meta-imagination and MDP-imagination (Markov-Decision-Process). We perform meta-imagination by interpolating on the learned latent context space with disentangled properties, as well as MDP-imagination through the generative world model where physical knowledge is added to plain VAE networks. Our experiments with various benchmarks show that MetaDreamer outperforms existing approaches in data efficiency and interpolated generalization.
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2024.3417114