Adaptive Path-Tracking Controller Embedded With Reinforcement Learning and Preview Model for Autonomous Driving

Path tracking control is a crucial function in an autonomous vehicle. Previous studies have presented conventional model-based controllers, which lack generalization ability, especially in dealing with high-curvature cases. In contrast to those model-based controllers, data-driven controllers, repre...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on vehicular technology 2024-12, p.1-15
Main Authors: Xia, Qi, Chen, Peng, Xu, Guoyan, Sun, Haodong, Li, Liang, Yu, Guizhen
Format: Article
Language:English
Subjects:
Accuracy
Adaptation models
Adaptive systems
autonomous driving
Autonomous vehicles
Control theory
Neural networks
Planning
preview control
reinforcement learning
Tracking
Tracking control
Training
Vehicle dynamics
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Path tracking control is a crucial function in an autonomous vehicle. Previous studies have presented conventional model-based controllers, which lack generalization ability, especially in dealing with high-curvature cases. In contrast to those model-based controllers, data-driven controllers, represented by reinforcement learning (RL), are promising to improve tracking control accuracy and generalization by online learning. However, the inherent inexplicability of RL degrades the stability of an RL-based controller, further limiting its applications in real-world autonomous driving cases. To resolve this issue, this study proposes an adaptive tracking controller based on RL, allowing to enhance network stability through a preview model and adaptive correction. It is composed of three modules, namely a lateral controller, a lateral adaptive corrector, and a longitudinal speed planner. Specifically, the preview control theory is combined with the twin delayed deep deterministic policy gradient (TD3) method to improve the tracking convergence of the lateral controller; the adaptive corrector derives correction compensations based on predicted waypoints along a path, thus enhancing control accuracy; the longitudinal planner outputs a speed profile via a reward increment soft actor-critic algorithm, through which large lateral tracking errors are reduced. Using an intelligent vehicle with sufficient steering capability, simulations and real-world tests show that curvy paths with a curvature larger than 2.0 m −1 can be closely tracked within an error bound of 0.1 m.
ISSN:0018-9545
DOI:10.1109/TVT.2024.3502640