Flow forecasting for leakage burst prediction in water distribution systems using long short-term memory neural networks and Kalman filtering

• Published in: Sustainable cities and society 2023-12, Vol.99, p.104934, Article 104934
• Main Authors: McMillan, Lauren, Fayaz, Jawad, Varga, Liz
• Format: Article
• Language: English
• Subjects: Kalman filter; Leakage prediction; Long short-term memory; Recurrent neural networks; Self-healing systems; Water flow forecasting
• ISSN: 2210-6707; 2210-6715
• DOI: 10.1016/j.scs.2023.104934

•Hybrid Deep-Learning Framework for leakage and regular flow forecasting.•LSTM-RNN model for mean flow prediction.•Kalman filtering applied to update estimates using real-time fluctuations.•Framework is trained and tested using 12-month flow data from over 2000 DMAs.•The proposed FLUIDS framework achieves high levels of accuracy on unseen test data. Reducing pipe leakage is one of the top priorities for water companies, with many investing in higher quality sensor coverage to improve flow forecasting and detection of leaks. Most research on this topic is focused on leakage detection through the analysis of sensor data from district metered areas (DMAs), aiming to identify bursts after their occurrence. This study is a step towards the development of ‘self-healing’ water infrastructure systems. In particular, machine learning and deep learning-based algorithms are applied to forecasting the anomalous water flow experienced during bursts (new leakage) in DMAs at various temporal scales, thereby aiding in the health monitoring of water distribution systems. This study uses a dataset of over 2,000 DMAs in North Yorkshire, UK, containing flow time series recorded at 15-minute intervals for a period of one year. Firstly, the method of isolation forests is used to identify anomalies in the dataset, which are cross referenced with entries in the water mains repair log, indicating the occurrence of bursts. Going beyond leakage detection, this research proposes a hybrid deep learning framework named FLUIDS (Forecasting Leakage and Usual flow Intelligently in water Distribution Systems). A recurrent neural network (RNN) is used for mean flow forecasting, which is then combined with forecasted residuals obtained through real-time Kalman filtering. While providing expected day-to-day flow demands, this framework also aims to issue sufficient early warning for any upcoming anomalous flow or possible leakages. For a given forecast period, the FLUIDS framework can be used to compute the probability of flow exceeding a pre-defined threshold, thus allowing decision-making for any necessary interventions. This can inform targeted repair strategies that best utilize resources to minimize leakages and disruptions. The FLUIDS framework is statistically assessed and compared against the state-of-practice minimum night flow (MNF) methodology. Based on the statistical analyses, it is concluded that the proposed framework performs well on the unobserved test dataset for both regula