Harnessing feature engineering to improve machine learning: A review of different data processing techniques
Sabri My Abdelouahed, Rahmouni Abla, Ennaji Asmae, Aarab Abdellah
Abstract
This study delves into the critical realm of feature engineering, examining its influence on the performance of various machine learning classifiers under different data preprocessing conditions. Our research encompasses an array of algorithms, including Support Vector Machine (SVM), Random Forest, Decision Trees, Artificial Neural Networks (ANN), and K-Nearest Neighbors (KNN). A key aspect of our methodology is the employment of Grid Search to systematically explore a wide range of hyperparameter configurations for each classifier. Focusing on handling missing values, outliers, and data scaling, we assess the impact of these feature engineering techniques on model performance. The evaluation is based on metrics such as accuracy, recall, precision, and F1-score, providing a comprehensive comparative analysis. The results offer insightful revelations about the differential impacts of feature engineering strategies on each algorithm’s predictive accuracy and overall effectiveness. In particular, the study highlights how specific algorithms, notably Random Forest demonstrate robust performance across varied data scenarios. These findings contribute significantly to the understanding of feature engineering in machine learning, offering practical guidance for selecting and optimizing data preprocessing methods. This research underscores the paramount importance of fine-tuning feature engineering approaches and lays a foundation for future advancements in optimizing machine learning models.