Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications

• Published in: Energies (Basel) 2023-07, Vol.16 (14), p.5326
• Main Authors: Vamvakas, Dimitrios, Michailidis, Panagiotis, Korkas, Christos, Kosmatopoulos, Elias
• Format: Article
• Language: English
• Subjects: Alternative energy sources; Artificial intelligence; Battery chargers; building energy management; Computational linguistics; Control algorithms; Cost control; deep learning; Electric vehicle charging stations; Electric vehicles; Emissions (Pollution); Energy consumption; Energy efficiency; Energy resources; Environmental impact; Hierarchical Reinforcement Learning (HRL); HVAC; Language processing; Machine Learning; Mathematical models; Methods; Natural language interfaces; Optimization; power and energy systems; Reinforcement Learning (RL); Renewable resources; Weather forecasting
• ISSN: 1996-1073; 1996-1073
• DOI: 10.3390/en16145326

With the rise in electricity, gas and oil prices and the persistently high levels of carbon emissions, there is an increasing demand for effective energy management in energy systems, including electrical grids. Recent literature exhibits large potential for optimizing the behavior of such systems towards energy performance, reducing peak loads and exploiting environmentally friendly ways for energy production. However, the primary challenge relies on the optimization of such systems, which introduces significant complexities since they present quite dynamic behavior. Such cyberphysical frameworks usually integrate multiple interconnected components such as power plants, transmission lines, distribution networks and various types of energy-storage systems, while the behavior of these components is affected by various external factors such as user individual requirements, weather conditions, energy demand and market prices. Consequently, traditional optimal control approaches—such as Rule-Based Control (RBC)—prove inadequate to deal with the diverse dynamics which define the behavior of such complicated frameworks. Moreover, even sophisticated techniques—such as Model Predictive Control (MPC)—showcase model-related limitations that hinder the applicability of an optimal control scheme. To this end, AI model-free techniques such as Reinforcement Learning (RL) offer a fruitful potential for embedding efficient optimal control in cases of energy systems. Recent studies present promising results in various fields of engineering, indicating that RL frameworks may prove the key element for delivering efficient optimal control in smart buildings, electric vehicle charging and smart grid applications. The current paper provides a comprehensive review of RL implementations in energy systems frameworks—such as Renewable Energy Sources (RESs), Building Energy-Management Systems (BEMSs) and Electric Vehicle Charging Stations (EVCSs)—illustrating the benefits and the opportunities of such approaches. The work examines more than 80 highly cited papers focusing on recent RL research applications—between 2015 and 2023—and analyzes the model-free RL potential as regards the energy systems’ control optimization in the future.