Loading…

Improving Car-Following Control in Mixed Traffic: A Deep Reinforcement Learning Framework with Aggregated Human-Driven Vehicles

Traffic oscillations pose safety and efficiency challenges in mixed scenarios involving connected and automated vehicles (CAVs) and human-driven vehicles (HDVs). Existing control strategies fail to handle the unpredictability of HDV behaviors, resulting in disruptive "stop-and-go" traffic...

Full description

Saved in:
Bibliographic Details
Main Authors: Chen, Xianda, Tiu, PakHin, Zhang, Yihuai, Zhu, Meixin, Zheng, Xinhu, Wang, Yinhai
Format: Conference Proceeding
Language:English
Subjects:
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page 632
container_issue
container_start_page 627
container_title
container_volume
creator Chen, Xianda
Tiu, PakHin
Zhang, Yihuai
Zhu, Meixin
Zheng, Xinhu
Wang, Yinhai
description Traffic oscillations pose safety and efficiency challenges in mixed scenarios involving connected and automated vehicles (CAVs) and human-driven vehicles (HDVs). Existing control strategies fail to handle the unpredictability of HDV behaviors, resulting in disruptive "stop-and-go" traffic patterns. This study proposes a novel algorithm that uses Deep Reinforcement Learning (DRL) integrated into a distinctive "CAV-AHDV-CAV" structure for car-following events. The consecutive HDVs are treated as an aggregated unit called Aggregated HDVs (AHDVs) to eliminate stochasticity and leverage collective traffic features as inputs, addressing the driver heterogeneity issue. Our training and testing were conducted using a dataset of 9,200 car-following events extracted from the HighD dataset. In these events, the lead vehicle serves as our CAV in front, while the following vehicle represents the AHDV. We simulated our controlled vehicle to follow the AHDV, aiming to achieve the vehicle equilibrium state with respect to both the AHDV and the CAV in front. The results demonstrate a reduction in the impact of HDVs and an enhancement of equilibrium states compared to baseline models. Specifically, we achieved a speed mean square error (MSE) of 3.151 and spacing MSE values of 50.484 (with respect to the AHDV) and 47.855 (with respect to the CAV). These findings offer robust and adaptable control strategies for efficient and safe mixed traffic dominated by CAVs.
doi_str_mv 10.1109/IV55156.2024.10588405
format conference_proceeding
fullrecord <record><control><sourceid>ieee_CHZPO</sourceid><recordid>TN_cdi_ieee_primary_10588405</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10588405</ieee_id><sourcerecordid>10588405</sourcerecordid><originalsourceid>FETCH-LOGICAL-i106t-93b1eb3407f267ec6689554b2a73053816f6db7c1dde58821f88793710f8a8133</originalsourceid><addsrcrecordid>eNo1kMtOAjEYRquJiYi8gSZ9gcH-7fQy7giIkGBMDLIlnZm_Q3WmQ8oIuvLVxdvqy1l8Z3EIuQY2BGDZzXwlJUg15IynQ2DSmJTJEzLIdGaEZCI1BuCU9LhKeaI5pOfkYrd7YUxKzqFHPufNNrZ7Hyo6tjGZtnXdHn6oDV1sa-oDffDvWNJltM754paO6ARxS5_QB9fGAhsMHV2gjeH7N422wUMbX-nBdxs6qqqIle2OgtlbY0MyiX6Pga5w44sad5fkzNl6h4O_7ZPn6d1yPEsWj_fz8WiReGCqSzKRA-YiZdpxpbFQymRSpjm3WjApDCinylwXUJZ4bMDBGaMzoYE5Yw0I0SdXv16PiOtt9I2NH-v_YOILkjZf7w</addsrcrecordid><sourcetype>Publisher</sourcetype><iscdi>true</iscdi><recordtype>conference_proceeding</recordtype></control><display><type>conference_proceeding</type><title>Improving Car-Following Control in Mixed Traffic: A Deep Reinforcement Learning Framework with Aggregated Human-Driven Vehicles</title><source>IEEE Xplore All Conference Series</source><creator>Chen, Xianda ; Tiu, PakHin ; Zhang, Yihuai ; Zhu, Meixin ; Zheng, Xinhu ; Wang, Yinhai</creator><creatorcontrib>Chen, Xianda ; Tiu, PakHin ; Zhang, Yihuai ; Zhu, Meixin ; Zheng, Xinhu ; Wang, Yinhai</creatorcontrib><description>Traffic oscillations pose safety and efficiency challenges in mixed scenarios involving connected and automated vehicles (CAVs) and human-driven vehicles (HDVs). Existing control strategies fail to handle the unpredictability of HDV behaviors, resulting in disruptive "stop-and-go" traffic patterns. This study proposes a novel algorithm that uses Deep Reinforcement Learning (DRL) integrated into a distinctive "CAV-AHDV-CAV" structure for car-following events. The consecutive HDVs are treated as an aggregated unit called Aggregated HDVs (AHDVs) to eliminate stochasticity and leverage collective traffic features as inputs, addressing the driver heterogeneity issue. Our training and testing were conducted using a dataset of 9,200 car-following events extracted from the HighD dataset. In these events, the lead vehicle serves as our CAV in front, while the following vehicle represents the AHDV. We simulated our controlled vehicle to follow the AHDV, aiming to achieve the vehicle equilibrium state with respect to both the AHDV and the CAV in front. The results demonstrate a reduction in the impact of HDVs and an enhancement of equilibrium states compared to baseline models. Specifically, we achieved a speed mean square error (MSE) of 3.151 and spacing MSE values of 50.484 (with respect to the AHDV) and 47.855 (with respect to the CAV). These findings offer robust and adaptable control strategies for efficient and safe mixed traffic dominated by CAVs.</description><identifier>EISSN: 2642-7214</identifier><identifier>EISBN: 9798350348811</identifier><identifier>DOI: 10.1109/IV55156.2024.10588405</identifier><language>eng</language><publisher>IEEE</publisher><subject>Aggregated HDVs ; Connected and Automated Vehicles ; Deep reinforcement learning ; Feature extraction ; Human-Driven Vehicles ; Intelligent vehicles ; Lead ; Mean square error methods ; Mixed Traffic ; Traffic control ; Traffic Oscillations ; Training</subject><ispartof>2024 IEEE Intelligent Vehicles Symposium (IV), 2024, p.627-632</ispartof><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10588405$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>309,310,780,784,789,790,27924,54554,54931</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/10588405$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Chen, Xianda</creatorcontrib><creatorcontrib>Tiu, PakHin</creatorcontrib><creatorcontrib>Zhang, Yihuai</creatorcontrib><creatorcontrib>Zhu, Meixin</creatorcontrib><creatorcontrib>Zheng, Xinhu</creatorcontrib><creatorcontrib>Wang, Yinhai</creatorcontrib><title>Improving Car-Following Control in Mixed Traffic: A Deep Reinforcement Learning Framework with Aggregated Human-Driven Vehicles</title><title>2024 IEEE Intelligent Vehicles Symposium (IV)</title><addtitle>IV</addtitle><description>Traffic oscillations pose safety and efficiency challenges in mixed scenarios involving connected and automated vehicles (CAVs) and human-driven vehicles (HDVs). Existing control strategies fail to handle the unpredictability of HDV behaviors, resulting in disruptive "stop-and-go" traffic patterns. This study proposes a novel algorithm that uses Deep Reinforcement Learning (DRL) integrated into a distinctive "CAV-AHDV-CAV" structure for car-following events. The consecutive HDVs are treated as an aggregated unit called Aggregated HDVs (AHDVs) to eliminate stochasticity and leverage collective traffic features as inputs, addressing the driver heterogeneity issue. Our training and testing were conducted using a dataset of 9,200 car-following events extracted from the HighD dataset. In these events, the lead vehicle serves as our CAV in front, while the following vehicle represents the AHDV. We simulated our controlled vehicle to follow the AHDV, aiming to achieve the vehicle equilibrium state with respect to both the AHDV and the CAV in front. The results demonstrate a reduction in the impact of HDVs and an enhancement of equilibrium states compared to baseline models. Specifically, we achieved a speed mean square error (MSE) of 3.151 and spacing MSE values of 50.484 (with respect to the AHDV) and 47.855 (with respect to the CAV). These findings offer robust and adaptable control strategies for efficient and safe mixed traffic dominated by CAVs.</description><subject>Aggregated HDVs</subject><subject>Connected and Automated Vehicles</subject><subject>Deep reinforcement learning</subject><subject>Feature extraction</subject><subject>Human-Driven Vehicles</subject><subject>Intelligent vehicles</subject><subject>Lead</subject><subject>Mean square error methods</subject><subject>Mixed Traffic</subject><subject>Traffic control</subject><subject>Traffic Oscillations</subject><subject>Training</subject><issn>2642-7214</issn><isbn>9798350348811</isbn><fulltext>true</fulltext><rsrctype>conference_proceeding</rsrctype><creationdate>2024</creationdate><recordtype>conference_proceeding</recordtype><sourceid>6IE</sourceid><recordid>eNo1kMtOAjEYRquJiYi8gSZ9gcH-7fQy7giIkGBMDLIlnZm_Q3WmQ8oIuvLVxdvqy1l8Z3EIuQY2BGDZzXwlJUg15IynQ2DSmJTJEzLIdGaEZCI1BuCU9LhKeaI5pOfkYrd7YUxKzqFHPufNNrZ7Hyo6tjGZtnXdHn6oDV1sa-oDffDvWNJltM754paO6ARxS5_QB9fGAhsMHV2gjeH7N422wUMbX-nBdxs6qqqIle2OgtlbY0MyiX6Pga5w44sad5fkzNl6h4O_7ZPn6d1yPEsWj_fz8WiReGCqSzKRA-YiZdpxpbFQymRSpjm3WjApDCinylwXUJZ4bMDBGaMzoYE5Yw0I0SdXv16PiOtt9I2NH-v_YOILkjZf7w</recordid><startdate>20240602</startdate><enddate>20240602</enddate><creator>Chen, Xianda</creator><creator>Tiu, PakHin</creator><creator>Zhang, Yihuai</creator><creator>Zhu, Meixin</creator><creator>Zheng, Xinhu</creator><creator>Wang, Yinhai</creator><general>IEEE</general><scope>6IE</scope><scope>6IL</scope><scope>CBEJK</scope><scope>RIE</scope><scope>RIL</scope></search><sort><creationdate>20240602</creationdate><title>Improving Car-Following Control in Mixed Traffic: A Deep Reinforcement Learning Framework with Aggregated Human-Driven Vehicles</title><author>Chen, Xianda ; Tiu, PakHin ; Zhang, Yihuai ; Zhu, Meixin ; Zheng, Xinhu ; Wang, Yinhai</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-i106t-93b1eb3407f267ec6689554b2a73053816f6db7c1dde58821f88793710f8a8133</frbrgroupid><rsrctype>conference_proceedings</rsrctype><prefilter>conference_proceedings</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Aggregated HDVs</topic><topic>Connected and Automated Vehicles</topic><topic>Deep reinforcement learning</topic><topic>Feature extraction</topic><topic>Human-Driven Vehicles</topic><topic>Intelligent vehicles</topic><topic>Lead</topic><topic>Mean square error methods</topic><topic>Mixed Traffic</topic><topic>Traffic control</topic><topic>Traffic Oscillations</topic><topic>Training</topic><toplevel>online_resources</toplevel><creatorcontrib>Chen, Xianda</creatorcontrib><creatorcontrib>Tiu, PakHin</creatorcontrib><creatorcontrib>Zhang, Yihuai</creatorcontrib><creatorcontrib>Zhu, Meixin</creatorcontrib><creatorcontrib>Zheng, Xinhu</creatorcontrib><creatorcontrib>Wang, Yinhai</creatorcontrib><collection>IEEE Electronic Library (IEL) Conference Proceedings</collection><collection>IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume</collection><collection>IEEE Xplore All Conference Proceedings</collection><collection>IEEE Electronic Library (IEL)</collection><collection>IEEE Proceedings Order Plans (POP All) 1998-Present</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Chen, Xianda</au><au>Tiu, PakHin</au><au>Zhang, Yihuai</au><au>Zhu, Meixin</au><au>Zheng, Xinhu</au><au>Wang, Yinhai</au><format>book</format><genre>proceeding</genre><ristype>CONF</ristype><atitle>Improving Car-Following Control in Mixed Traffic: A Deep Reinforcement Learning Framework with Aggregated Human-Driven Vehicles</atitle><btitle>2024 IEEE Intelligent Vehicles Symposium (IV)</btitle><stitle>IV</stitle><date>2024-06-02</date><risdate>2024</risdate><spage>627</spage><epage>632</epage><pages>627-632</pages><eissn>2642-7214</eissn><eisbn>9798350348811</eisbn><abstract>Traffic oscillations pose safety and efficiency challenges in mixed scenarios involving connected and automated vehicles (CAVs) and human-driven vehicles (HDVs). Existing control strategies fail to handle the unpredictability of HDV behaviors, resulting in disruptive "stop-and-go" traffic patterns. This study proposes a novel algorithm that uses Deep Reinforcement Learning (DRL) integrated into a distinctive "CAV-AHDV-CAV" structure for car-following events. The consecutive HDVs are treated as an aggregated unit called Aggregated HDVs (AHDVs) to eliminate stochasticity and leverage collective traffic features as inputs, addressing the driver heterogeneity issue. Our training and testing were conducted using a dataset of 9,200 car-following events extracted from the HighD dataset. In these events, the lead vehicle serves as our CAV in front, while the following vehicle represents the AHDV. We simulated our controlled vehicle to follow the AHDV, aiming to achieve the vehicle equilibrium state with respect to both the AHDV and the CAV in front. The results demonstrate a reduction in the impact of HDVs and an enhancement of equilibrium states compared to baseline models. Specifically, we achieved a speed mean square error (MSE) of 3.151 and spacing MSE values of 50.484 (with respect to the AHDV) and 47.855 (with respect to the CAV). These findings offer robust and adaptable control strategies for efficient and safe mixed traffic dominated by CAVs.</abstract><pub>IEEE</pub><doi>10.1109/IV55156.2024.10588405</doi><tpages>6</tpages></addata></record>
fulltext fulltext_linktorsrc
identifier EISSN: 2642-7214
ispartof 2024 IEEE Intelligent Vehicles Symposium (IV), 2024, p.627-632
issn 2642-7214
language eng
recordid cdi_ieee_primary_10588405
source IEEE Xplore All Conference Series
subjects Aggregated HDVs
Connected and Automated Vehicles
Deep reinforcement learning
Feature extraction
Human-Driven Vehicles
Intelligent vehicles
Lead
Mean square error methods
Mixed Traffic
Traffic control
Traffic Oscillations
Training
title Improving Car-Following Control in Mixed Traffic: A Deep Reinforcement Learning Framework with Aggregated Human-Driven Vehicles
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-13T03%3A31%3A25IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-ieee_CHZPO&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=proceeding&rft.atitle=Improving%20Car-Following%20Control%20in%20Mixed%20Traffic:%20A%20Deep%20Reinforcement%20Learning%20Framework%20with%20Aggregated%20Human-Driven%20Vehicles&rft.btitle=2024%20IEEE%20Intelligent%20Vehicles%20Symposium%20(IV)&rft.au=Chen,%20Xianda&rft.date=2024-06-02&rft.spage=627&rft.epage=632&rft.pages=627-632&rft.eissn=2642-7214&rft_id=info:doi/10.1109/IV55156.2024.10588405&rft.eisbn=9798350348811&rft_dat=%3Cieee_CHZPO%3E10588405%3C/ieee_CHZPO%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-i106t-93b1eb3407f267ec6689554b2a73053816f6db7c1dde58821f88793710f8a8133%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rft_ieee_id=10588405&rfr_iscdi=true