Water Age Control for Water Distribution Networks via Safe Reinforcement Learning

• Published in: IEEE transactions on control systems technology 2024-11, Vol.32 (6), p.2332-2343
• Main Authors: Val Ledesma, Jorge, Wisniewski, Rafal, Kallesoe, Carsten S., Tsouvalas, Agisilaos
• Format: Article
• Language: English
• Subjects: Algorithms; Control algorithms; Control systems; Control theory; Distribution networks; Gaussian process; Gaussian process (GP); Industrial applications; Machine learning; Optimization; Regression models; Reinforcement learning; reinforcement learning (RL); Reservoirs; Safety; Safety management; Subspace constraints; water age; Water distribution; water distribution network (WDN); Water engineering; Water quality
• ISSN: 1063-6536; 1558-0865
• DOI: 10.1109/TCST.2024.3426300

Reinforcement learning (RL) is a widely used control technique that finds an optimal policy using the feedback of its actions. The search for the optimal policy requires that the system explores a broad region of the state space. This search puts at risk the safe operation, since some of the explored regions might be near the physical system limits. Implementing learning methods in industrial applications is limited because of its uncertain behavior when finding an optimal policy. This work proposes an RL control algorithm with a filter that supervises the safety of the exploration based on a nominal model. The performance of this safety filter is increased by modeling the uncertainty with a Gaussian process (GP) regression. This method is applied to the optimization of the management of a water distribution network (WDN) with an elevated reservoir; the management objectives are to regulate the tank filling while maintaining an adequate water turnover. The proposed method is validated in a laboratory setup that emulates the hydraulic features of a WDN.