Combined Sewer Overflow and Flooding Mitigation Through a Reliable Real‐Time Control Based on Multi‐Reinforcement Learning and Model Predictive Control

Abstract Real‐time control (RTC) of urban drainage systems (UDS) has been proved an efficient tool in combined sewer overflow (CSO) and flooding mitigation. Recently, new RTC approaches based on reinforcement learning (RL) were developed for flooding mitigation in stormwater systems. While these studies have made contributions to enable an improved urban water management, they are insufficient to allow for deeply understanding of the effectiveness of different RLs in UDS. Meanwhile, the risk of handing over the control process to a RL agent is still unavoidable because of the fluctuations of RLs' output and the unknown consequences of implementing RLs control strategy. This study conducted four tasks to address these problems. First, five RTC systems based on five individual RLs were designed to distinguish different RLs' performance in the context of UDS. Then, an independent security system based on SWMM was provided to forecast and evaluate the consequence of RL control strategy. After that, an innovative hybrid RTC system, called Voting, was developed by coupling multiple RLs and the independent security system through a model predictive control framework to avoid the fluctuations of RLs' output. Finally, the robustness of the RL agents was validated using uncertainty analysis. All the RLs were evaluated through simulation based on a Storm Water Management Model of a UDS located in Eastern China. According to the results, (a) different RLs show promise in CSO and flooding mitigation; (b) Voting selects a relatively reliable and optimal control trajectory compared with any single RL agents; (c) the performances of RL agents have certain robustness when facing different rainfall events and imperfect input.