The Impact of Feature Scaling in Machine Learning: Effects on Regression and Classification Tasks
João Manoel Herrera Pinheiro, Suzana Vilas Boas de Oliveira, Thiago Silva, Pedro Saraiva, Ênio Pereira de Souza, Ricardo V. Godoy, Leonardo A. Ambrósio, Marcelo Becker
Abstract
This study addresses the lack of comprehensive evaluations of feature scaling by systematically assessing 12 techniques, including less common methods such as VAST and Pareto, in 14 machine learning algorithms and 16 datasets covering both classification and regression tasks. The impact of feature scaling was evaluated in terms of predictive performance (accuracy, MAE, MSE, R²) and computational efficiency (training time, inference time, memory usage). The results show that the ensemble methods (Random Forest, XGBoost, CatBoost, LightGBM) remain robust regardless of scaling, while models such as Logistic Regression, SVM, TabNet, and MLP are highly sensitive to the chosen scaler. By making all codes, results, and model parameters publicly available, this work provides reproducible, model-specific guidance for selecting scaling strategies in practical machine learning applications.