Reinforcement Learning Based on Tree Model Environment in Automatic Regulation of Water Treatment Dosing

Machine learning advancements have facilitated the intelligent transformation of wastewater treatment plants (WWTPs), enabling the deployment of predictive models to optimize operational performance. Current implementations predominantly utilize tree-based algorithms and neural networks that require continuous parameter recalibration to adapt to dynamic operational conditions. However, existing research on reinforcement learning applications in intelligent WWTP management remains largely confined to simulation environments, particularly the Benchmark Simulation Model No. 1 (BSM1), which fails to adequately capture the complexity and variability of real-world operational scenarios. This study addresses phosphorus removal dosing control by proposing the utilization of a Gradient Boosting Decision Tree (GBDT) model trained on real plant data as the simulation environment for agent interaction. Two reinforcement learning algorithms, Proximal Policy Optimization (PPO) and Twin Delayed Deep Deterministic Policy Gradient (TD3), were trained and their performance compared against the original GBDT dosing control model under operational conditions. Results show that PPO achieves a 17.1% reduction in chemical dosing relative to the baseline system, while TD3 attains a 15.9% reduction, with both maintaining equivalent effluent quality standards. These findings indicate that employing tree-based models trained on actual plant data as training environments could accelerate agent learning while providing a superior representation of real-world conditions. Given the transferable nature of reinforcement learning, this approach presents potential for fully autonomous WWTP operations without requiring frequent parameter adjustments.