Automated Feature Engineering and Hyperparameter optimization for Machine Learning
Mihir Gada, Zenil Haria, Arnav Mankad, Kaustubh Damania, Smita Sankhe
Abstract
Machine learning (ML) has proven to be important in many parts of our lives and has been beneficial to us. Nevertheless, designing a highly performant Machine Learning system for a specific application requires high knowledge and proficiency, thus hindering its application in many areas. Data pre-processing, feature engineering, and hyperparameter optimization influence the performance of models immensely, thus automating the same will reduce the burden of amateur users and save time for a lot of Machine Learning experts who are experimenting. In this paper, we implement the methods for automating the data pre-processing, feature processing, and hyperparameter optimization. The methods are exclusively implemented for tabular data which is in the textual formal used for supervised learning. We evaluate, compare and summarize the results on the basis of execution time and various evaluation metrics for the respective tasks. Finally, the best set of features and hyperparameters for the model to train on are returned. The overall performance of combinations of hyperparameter optimization and feature processing techniques has been evaluated on various datasets. With the help of the results, the most optimum technique for feature engineering and hyperparameter optimization are deduced.