Flooding mitigation through safe & trustworthy reinforcement learning

• Published in: Journal of hydrology (Amsterdam) 2023-05, Vol.620, p.129435, Article 129435
• Main Authors: Tian, Wenchong, Xin, Kunlun, Zhang, Zhiyu, Zhao, Muhan, Liao, Zhenliang, Tao, Tao
• Format: Article
• Language: English
• Subjects: Flooding mitigation; Reinforcement learning; Safe learning; Uncertainty analysis; Urban drainage system
• ISSN: 0022-1694
• DOI: 10.1016/j.jhydrol.2023.129435

•RLs’ safety in the context of UDS is investigated through a statistic model.•RLs’ safety in the context of UDS is improved by safe learning methods.•RLs’ robustness is influenced by rainfalls, perturbation, the type of RL. Reinforcement learning (RL) has been used in real-time control of urban drainage system (UDS) for flooding mitigation, achieving a milestone in urban water management. However, RL can only guarantee an optimization control, rather than keep the control trajectory safe and trustworthy. Therefore, unacceptable risk still exists when handing over the real-world control process to an RL agent. Although safe learning is effective in enhancing RL’s safety, it cannot be applied directly due to the lack of quantitative framework of RL’s safety in UDS context. This study conducts three tasks to investigate and improve the safety of RL in UDS. First, a metric framework of RLs’ safety in the context of UDS is provided through a mathematic description. Then, it is plugged into safe learning methods to improve RLs’ safety in UDS. After that, a systemic uncertainty analysis is employed to evaluate the robustness of RL. The results of the case study indicate that (i) all the RLs show a promising result in flooding mitigation; (ii) safe learning helps RLs achieve a safer control process with a lower average water level and lower frequency of orifices operation; (iii) the robustness of RLs in UDS is influenced by the volume of rainfalls, the degree of randomness, and the type of RLs.