Effects of feature selection methods in estimating SO2 concentration variations using machine learning and stacking ensemble approach

Abstract

Statistical-based feature selection methods have been used for dimension reduction, but only a few studies have explored the impact of selected features on machine learning models. This study aims to investigate the effects of statistical and machine learning-based feature selection methods on spatial prediction models for estimating variations in SO 2 concentrations. We collected daily SO 2 observations from 1994 to 2018 along with predictor variables such as land-use/land cover allocations, roads, landmarks, meteorological factors, and satellite images, resulting in a total of 428 geographic predictors. Important features were identified using statistical-based feature selection methods including SelectKBest, stepwise feature selection, elastic net, and machine learning-based methods such as random forest. The selected features from the four feature selection methods were fitted to machine learning algorithms including gradient boosting, CatBoost, XGBoost, and stacking ensemble to establish prediction models for estimating SO 2 concentrations. SHapley Additive exPlanations (SHAP) was applied to explain the contribution of each selected feature to the model's prediction capability. The results showed that stacking ensemble model outperformed the three single machine learning algorithms. Among the four feature selection methods, the random forest method yielded the highest prediction accuracy (R 2 =0.80) in the training model, followed by stepwise selection (R 2 =0.75), SelectKBest (R 2 =0.75), and elastic net (R 2 =0.72) in the stacking ensemble model. These results were robust after several validation tests. Our findings suggested that the random forest feature selection method was more suitable for developing machine learning models for air pollution estimation. The identified features also provide important information for urban air pollution management. • Feature selection methods affect prediction accuracy of machine learning. • Random forest feature selection method outperformed the other methods. • Random forest improved 8 % of model prediction accuracy than stepwise selection. • Stacking ensemble learning model explained 80 % of SO 2 spatial variance. • Explainable features affecting SO 2 levels could be recognized by the random forest.

Topics & Concepts