A critical review on selecting performance evaluation metrics for supervised machine learning models in wastewater quality prediction

Abstract

While numerous machine learning (ML) models have been applied to wastewater treatment plant (WWTP) quality prediction tasks, significantly less attention has been paid to the selection and interpretation of performance evaluation metrics. Most studies rely on general-purpose regression metrics such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and the Coefficient of Determination (R 2 ). Although widely used, these metrics differ considerably in terms of interpretability, sensitivity to data anomalies, and suitability for dynamic, noisy environments like WWTPs. This review, based on a systematic literature analysis of 27 performance evaluation metrics, critically examines their theoretical foundations, strengths, limitations, and applicability within the context of supervised ML-based WWTP modeling. In addition to statistical metrics, it also explores complementary graphical techniques, such as residual or failure prediction plots, that offer deeper insights into model behavior, insights that purely numerical indicators may overlook. A significant contribution of this paper is the development of a practical, decision-guiding flowchart to assist researchers in selecting appropriate evaluation metrics based on dataset characteristics, modeling objectives, and project constraints. Additionally, it summarizes a reference toolkit of graphical methods that have been used in the literature to assess model performance beyond numerical indicators. Together, these resources not only promote more informed and transparent metric selection in research but also provide wastewater practitioners with actionable tools for interpreting model outputs, comparing predictive approaches, and identifying the models most suitable for reliable process monitoring and operational decision-making. • Error metrics based on absolute differences are more favorable than squared ones. • Dimensionless metrics (ratio or normalized) should prioritize absolute differences. • R² can be deceptive when applied to nonlinear models; recommend using alternative ones. • Multiple metrics can capture different aspects of the model's performance. • Quantitative and visualization analyses help effectively evaluate the model performance.

Topics & Concepts