TD3-Based EMS Using Action Mask and Considering Battery Aging for Hybrid Electric Dump Trucks

The hybrid electric dump truck is equipped with multiple power sources, and each powertrain component is controlled by an energy management strategy (EMS) to split the demanded power. This study proposes an EMS based on deep reinforcement learning (DRL) algorithm to extend the battery life and reduc...

Full description

Saved in:
Bibliographic Details
Published in:World electric vehicle journal 2023-03, Vol.14 (3), p.74
Main Authors: Mo, Jinchuan, Yang, Rong, Zhang, Song, Zhou, Yongjian, Huang, Wei
Format: Article
Language:English
Subjects:
action mask
Aging
Algorithms
battery aging
deep reinforcement learning
Driving
Dump trucks
Efficiency
Electric vehicles
Energy consumption
Energy management
energy management strategy
hybrid electric dump truck
Hybrid systems
Machine learning
Objective function
Optimization
Power management
Power sources
Powertrain
Temperature
Trucks
Velocity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The hybrid electric dump truck is equipped with multiple power sources, and each powertrain component is controlled by an energy management strategy (EMS) to split the demanded power. This study proposes an EMS based on deep reinforcement learning (DRL) algorithm to extend the battery life and reduced total usage cost for the vehicle, namely the twin delayed deep deterministic policy gradient (TD3) based EMS. Firstly, the vehicle model is constructed and the optimization objective function, including battery aging cost and fuel consumption cost, is designed. Secondly, the TD3-based EMS is used for continuous action control of ICE power based on vehicle state, and the action mask is applied to filter out invalid actions. Thirdly, the simulations of the EMSs are trained under the CHTC-D driving cycle and C-WTVC driving cycle. The results show that the action mask improves the convergence efficiency of the strategies, and the proposed TD3-based EMS outperforms the deep deterministic policy gradient (DDPG) based EMS. Meanwhile, the battery life is extended by 36.17% under CHTC-D and 35.49% under C-WTVC, and the total usage cost is reduced by 4.30% and 2.49% when the EMS considers battery aging. In summary, the proposed TD3-based EMS can extend the battery life and reduce usage cost, and provides a method to solve the optimization problem for the EMS of hybrid power systems.
ISSN:2032-6653
2032-6653
DOI:10.3390/wevj14030074