SEAL: SEmantic-Augmented Imitation Learning via Language Model

• Published in: arXiv.org 2024-10
• Main Authors: Gu, Chengyang, Pan, Yuxin, Bai, Haotian, Xiong, Hui, Chen, Yize
• Format: Article
• Language: English
• Subjects: Hierarchies; Knowledge representation; Labels; Large language models; Semantics; Task complexity
• ISSN: 2331-8422

Hierarchical Imitation Learning (HIL) is a promising approach for tackling long-horizon decision-making tasks. While it is a challenging task due to the lack of detailed supervisory labels for sub-goal learning, and reliance on hundreds to thousands of expert demonstrations. In this work, we introduce SEAL, a novel framework that leverages Large Language Models (LLMs)'s powerful semantic and world knowledge for both specifying sub-goal space and pre-labeling states to semantically meaningful sub-goal representations without prior knowledge of task hierarchies. SEAL employs a dual-encoder structure, combining supervised LLM-guided sub-goal learning with unsupervised Vector Quantization (VQ) for more robust sub-goal representations. Additionally, SEAL incorporates a transition-augmented low-level planner for improved adaptation to sub-goal transitions. Our experiments demonstrate that SEAL outperforms state-of-the-art HIL methods and LLM-based planning approaches, particularly in settings with small expert datasets and complex long-horizon tasks.