BeTAIL: Behavior Transformer Adversarial Imitation Learning From Human Racing Gameplay

Autonomous racing poses a significant challenge for control, requiring planning minimum-time trajectories under uncertain dynamics and controlling vehicles at their handling limits. Current methods requiring hand-designed physical models or reward functions specific to each car or track. In contrast...

Full description

Saved in:
Bibliographic Details
Published in:IEEE robotics and automation letters 2024-08, Vol.9 (8), p.7302-7309
Main Authors: Weaver, Catherine, Tang, Chen, Hao, Ce, Kawamoto, Kenta, Tomizuka, Masayoshi, Zhan, Wei
Format: Article
Language:English
Subjects:
Adaptation models
Behavioral sciences
Decision making
Deep learning methods
Environment models
Human motion
imitation learning
Learning
Racing
reinforcement learning
Robotics
Task complexity
Training
Trajectory
Trajectory control
Trajectory planning
Transformers
Vehicle dynamics
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Autonomous racing poses a significant challenge for control, requiring planning minimum-time trajectories under uncertain dynamics and controlling vehicles at their handling limits. Current methods requiring hand-designed physical models or reward functions specific to each car or track. In contrast, imitation learning uses only expert demonstrations to learn a control policy. Imitated policies must model complex environment dynamics and human decision-making. Sequence modeling is highly effective in capturing intricate patterns of motion sequences but struggles to adapt to new environments or distribution shifts that are common in real-world robotics tasks. In contrast, Adversarial Imitation Learning (AIL) can mitigate this effect, but struggles with sample inefficiency and handling complex motion patterns. Thus, we propose BeTAIL: Behavior Transformer Adversarial Imitation Learning, which combines a Behavior Transformer (BeT) policy from human demonstrations with online AIL. BeTAIL adds an AIL residual policy to the BeT policy to model the sequential decision-making process of human experts and correct for out-of-distribution states or shifts in environment dynamics. We test BeTAIL on three challenges with expert-level demonstrations of real human gameplay in the high-fidelity racing game Gran Turismo Sport. Our proposed BeTAIL reduces environment interactions and improves racing performance and stability, even when the BeT is pretrained on different tracks than downstream learning.
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2024.3416789