Enhancing Cooperation of Vehicle Merging Control in Heavy Traffic Using Communication-Based Soft Actor-Critic Algorithm

A promising way to improve efficiency in highway on-ramp regions is to control connected and automated vehicles (CAVs) to pass the merging section sequentially. The primary objective of this paper is to incorporate the reinforcement learning (RL) technique into vehicle merging control to achieve glo...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on intelligent transportation systems 2023-06, Vol.24 (6), p.6491-6506
Main Authors: Li, Meng, Li, Zhibin, Wang, Shunchao, Zheng, Si
Format: Article
Language:English
Subjects:
Algorithms
Collision avoidance
Collision dynamics
communication protocol
Communications traffic
Cooperation
Driving conditions
Entropy
Merging
Merging strategy
mixed heavy traffic
Prediction algorithms
Protocols
Roads
soft actor-critic
Strategy
Traffic
Traffic conflicts
Traffic control
Traffic flow
Training
Trajectory
Travel time
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A promising way to improve efficiency in highway on-ramp regions is to control connected and automated vehicles (CAVs) to pass the merging section sequentially. The primary objective of this paper is to incorporate the reinforcement learning (RL) technique into vehicle merging control to achieve global cooperation. A communication protocol among RL agents is integrated with the Soft Actor-Critic (CSAC) algorithm. The parallel SAC agents with a parameter-sharing structure cooperate to optimize the common reward function. It enables CAVs to know the actions of each other so that they proactively negotiate to adjust speeds. We designed a parsimonious state representation containing crucial merging information to speed up the RL training. The effects of the merging control strategy were investigated with the simulation model in scenarios with different CAV penetration rates and traffic flow compositions. Results showed that the CSAC-based merging strategy generated collision-free merging trajectories with a short travel time while guaranteeing traffic safety in various traffic conditions. Four baselines (including two rule-based and two RL-based merging strategies) were applied in the same scenarios for comparison. It was found that the proposed merging strategy took the safest merging behaviors (zero traffic conflicts) and reduced the most significant travel time (56.9%).
ISSN:1524-9050
1558-0016
DOI:10.1109/TITS.2022.3221450