A multi-step predictive deep reinforcement learning algorithm for HVAC control systems in smart buildings

• Published in: Energy (Oxford) 2022-11, Vol.259, p.124857, Article 124857
• Main Authors: Liu, Xiangfei, Ren, Mifeng, Yang, Zhile, Yan, Gaowei, Guo, Yuanjun, Cheng, Lan, Wu, Chengke
• Format: Article
• Language: English
• Subjects: Deep reinforcement learning; Generalized correntropy; HVAC system; Multi-step prediction
• ISSN: 0360-5442
• DOI: 10.1016/j.energy.2022.124857

The development of the building energy management systems (BEMS) enable users to intelligently control Heating, Ventilation, Air-conditioning and Cooling (HVAC) systems based on digital information. In order to reduce the power consumption cost of the HVAC system while ensuring user satisfaction, a novel HVAC control system for building system based on a multi-step predictive deep reinforcement learning (MSP-DRL) algorithm is proposed in this paper. In the proposed method, the outdoor ambient temperature is predicted first by a featured deep learning method named GC-LSTM, where the Long Short-term Memory (LSTM) is enhanced by the generalized correntropy (GC) loss function to deal with the non-Gaussian characteristics of the collected outdoor temperature. In addition, the proposed temperature prediction model is combined with a reinforcement learning algorithm named Deep Deterministic Policy Gradient (DDPG) aiming to flexibly adjust the output power of the HVAC system under the dynamic changing of electricity prices. Finally, comprehensive simulation based on real world data is delivered. Numerical results show that the GC-LSTM algorithm is more accurate than other counterparts prediction algorithms, and the proposed HVAC control system based on the multi-step prediction deep reinforcement learning algorithm is effective and could save over 12% cost compared to other approaches, where the user comfort is maintained simultaneously. •A multi-step predictive deep reinforcement learning algorithm is proposed.•A GC-LSTM algorithm is adopted in multi-step outdoor temperature prediction.•A model-free DDPG algorithm is adopted to manage HVAC system for building system.•The proposed method performs well in the HVAC system.