Robust control and training risk reduction for boiler level control using two-stage training deep deterministic policy gradient

• Published in: Journal of the Taiwan Institute of Chemical Engineers 2022-01, Vol.130, p.103956, Article 103956
• Main Authors: Kang, Jia-Lin, Mirzaei, Somayeh, Zhou, Jia-An
• Format: Article
• Language: English
• Subjects: Boiler drum level control; Deep deterministic policy gradient; Inverse response curve; Multistep observation; Pretraining
• ISSN: 1876-1070; 1876-1089
• DOI: 10.1016/j.jtice.2021.06.050

•Offline pre-training of DDPG improved the performance of boiler level control.•2S-DDPG proved better control performance than DDPG and 3E control models.•Three-step 2S-DDPG showed the lowest IAE compared with DDPG and 3E control models.•2S-DDPG presented less fluctuation in the process with noise and time delay.•Three-step 2S-DDPG obtained a faster response time than DDPG and 3E control models. The stability of the boiler drum level is important for safe and stable operation of industrial plants. In this study, a two-stage training deep deterministic policy gradient (2S-DDPG) comprising offline pretraining and online training was proposed to control the boiler drum level. A comparison of simulation results between the 2S-DDPG, DDPG, and 3E training methods proved that 2S-DDPG can robustly control the boiler drum level. The 2S-DDPG model requires less than half as much interaction with online processes as DDPG does; this ensures stable industrial operation due to the lowered risk of process failures in training. The results indicated the integral absolute error of the three-step 2S-DDPG is the lowest among those of the three control models. Moreover, the three-step 2S-DDPG reduced the overshoot percentage calculated using 3E control from 59% to 0%. For processes with noise and time delay, 2S-DDPG exhibits a faster response and less variation in the control performance with regard to the boiler drum level. The manipulated variable distribution errors of the three-step 2S-DDPG were much less than those of the DDPG model. Therefore, 2S-DDPG can address the shortcomings of the traditional DDPG model.