A Brief Survey of Data Preprocessing in Machine Learning and Deep Learning Techniques
Bindu Bala, Sunny Behal
Abstract
Internet of Things (IoT) generates vast amounts of sensor data across various scientific and engineering domains. This raw data is often flawed and unsuitable for analysis due to noise, missing values, and outliers, potentially leading to inaccurate results. Thus, effective preprocessing techniques play a pivotal role in enhancing the efficiency and accuracy of large data. Artificial intelligence (AI) models are widely used for data analysis and interpretation across various domains like image classification, natural language processing, cyber-attack detection, and anomaly detection. So, the critical focus of preprocessing is to optimize AI modeling pipelines. By implementing appropriate preprocessing techniques on the required dataset, researchers can fully optimize the potential of machine learning (ML) and deep learning (DL). These techniques comprise data cleaning, normalization, feature selection, and dimensionality reduction to achieve smart, relevant, and consistent data from raw and unstructured data. This research study demonstrates a new taxonomy explaining various data pre-processing techniques utilized in ML and DL modeling and comparative analysis of such techniques being utilized in recent research studies which are useful to improve the quality of data and efficiency of the required dataset.