Task-Driven Autonomous Driving: Balanced Strategies Integrating Curriculum Reinforcement Learning and Residual Policy

• Published in: IEEE robotics and automation letters 2024-11, Vol.9 (11), p.9454-9461
• Main Authors: Shi, Jiamin, Zhang, Tangyike, Zong, Ziqi, Chen, Shitao, Xin, Jingmin, Zheng, Nanning
• Format: Article
• Language: English
• Subjects: autonomous driving; Autonomous vehicles; Curriculum learning; deep reinforcement learning; overtaking; Planning; residual policy; Safety; Task analysis; Training; Trajectory; Vehicle dynamics
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2024.3448237

Achieving fully autonomous driving in urban traffic scenarios is a significant challenge that necessitates balancing safety, efficiency, and compliance with traffic regulations. In this letter, we introduce a novel Curriculum Residual Hierarchical Reinforcement Learning (CR-HRL) framework. It integrates a rule-based planning model as a guiding mechanism, while a deep reinforcement learning algorithm generates supplementary residual strategies. This combination enables the RL agent to perform safe and efficient overtaking in complex traffic scenarios. Furthermore, we implement a detailed three-stage curriculum learning strategy that enhances the training process. By progressively increasing task complexity, the curriculum strategy effectively guides the exploration of autonomous vehicles and improves the reusability of sub-strategies. The effectiveness of the CR-HRL framework is confirmed through ablation experiments. Comparative experiments further highlight the superior efficiency and decision-making capabilities of our framework over traditional rule-based and RL baseline methods. Tests conducted with actual vehicles also demonstrate its practical applicability in real-world settings.