Reinforcement Learning for Real-Time Pricing and Scheduling Control in EV Charging Stations

• Published in: IEEE transactions on industrial informatics 2021-02, Vol.17 (2), p.849-859
• Main Authors: Wang, Shuoyao, Bi, Suzhi, Zhang, Yingjun Angela
• Format: Article
• Language: English
• Subjects: Algorithms; Approximation algorithms; Charging stations; Dynamic programming; Electric vehicle charging; Electric vehicle charging stations; Job shop scheduling; Learning; Linear functions; Load management; machine learning; Optimization; Pricing; pricing and scheduling; Real time; Real-time systems; reinforcement learning; Scheduling; state–action–reward–state–action (SARSA); Stochastic models
• ISSN: 1551-3203; 1941-0050
• DOI: 10.1109/TII.2019.2950809

This article proposes a reinforcement-learning (RL) approach for optimizing charging scheduling and pricing strategies that maximize the system objective of a public electric vehicle (EV) charging station. The proposed algorithm is "online" in the sense that the charging and pricing decisions made at each time depend only on the observation of past events, and is "model-free" in the sense that the algorithm does not rely on any assumed stochastic models of uncertain events. To cope with the challenge arising from the time-varying continuous state and action spaces in the RL problem, we first show that it suffices to optimize the total charging rates to fulfill the charging requests before departure times. Then, we propose a feature-based linear function approximator for the state-value function to further enhance the efficiency and generalization ability of the proposed algorithm. Through numerical simulations with real-world data, we show that the proposed RL algorithm achieves on average 138.5% higher charging-station profit than representative benchmark algorithms.