Noise Removal in the Presence of Significant Anomalies for Industrial IoT Sensor Data in Manufacturing
Yuehua Liu, Tharam S. Dillon, Wenjin Yu, Wenny Rahayu, Fahed Mostafa
Abstract
The emergence of the Industrial Internet of Things (IIoT) to enhance manufacturing and industrial processes allows data analysts to address significant problems such as predictive maintenance. For the purpose of accurate data analysis, cleansing noisy sensor data is one of the most fundamental and necessary steps. Without first removing the noise, the anomaly detection techniques are likely to give a large number of false positives. However, using traditional outlier detection methods directly for such analysis are not appropriate as both noise and significant anomalies might exist in the sensor data. This article introduces the new challenges and proposes a novel solution to address the issue of removing noise while preserving anomalies in the IIoT Data. It proposes an approach that measures both the rate of change and deviation to compute the noise score. It employs a sliding window technique to define the analysis unit of the contrast measure which is used in conjunction with statistical techniques. Extensive experiments demonstrate that the proposed approach outperforms the other state-of-the-art noise detection methods, providing a clean data set that preserves the anomalies on which one can effectively apply anomaly detection techniques.