Dynamic energy conversion and management strategy for an integrated electricity and natural gas system with renewable energy: Deep reinforcement learning approach

• Published in: Energy conversion and management 2020-09, Vol.220, p.113063, Article 113063
• Main Authors: Zhang, Bin, Hu, Weihao, Li, Jinghua, Cao, Di, Huang, Rui, Huang, Qi, Chen, Zhe, Blaabjerg, Frede
• Format: Article
• Language: English
• Subjects: Algorithms; Alternative energy sources; Conversion ratio; Decision making; Deep learning; Deep reinforcement learning; Dynamic energy conversion and management; Economic impact; Economics; Electric power generation; Electrical loads; Electricity; Electricity consumption; Energy conversion; Energy demand; Gas turbines; Information technology; Learning; Management; Markov processes; Multiple objective analysis; Natural gas; Optimization; Peak load; Reinforcement; Renewable energy; Renewable energy accommodation; Strategy; Wind power; Wind power generation
• ISSN: 0196-8904; 1879-2227
• DOI: 10.1016/j.enconman.2020.113063

•A DRL-based dynamic energy conversion and management model for IENGS is developed.•Uncertainty of customer’s demand and flexibility of wholesale prices are achieved.•Advanced deep reinforcement learning approach is applied. With the application of advanced information technology for the integration of electricity and natural gas systems, formulating an excellent energy conversion and management strategy has become an effective method to achieve established goals. Differing from previous works, this paper proposes a peak load shifting model to smooth the net load curve of an integrated electricity and natural gas system by coordinating the operations of the power-to-gas unit and generators. Moreover, the study aims to achieve multi-objective optimization while considering the economy of the system. A dynamic energy conversion and management strategy is proposed, which coordinates both the economic cost target and the peak load shifting target by adjusting an economic coefficient. To illustrate the complex energy conversion process, deep reinforcement learning is used to formulate the dynamic energy conversion and management problem as a discrete Markov decision process, and a deep deterministic policy gradient is adopted to solve the decision-making problem. By using the deep reinforcement learning method, the system operator can adaptively determine the conversion ratio of wind power, power-to-gas and gas turbine operations, and generator output through an online process, where the flexibility of wind power generation, wholesale gas price, and the uncertainties of energy demand are considered. Simulation results show that the proposed algorithm can increase the profit of the system operator, reduce wind power curtailment, and smooth the net load curves effectively in real time.