A Multihead Attention Self-Supervised Representation Model for Industrial Sensors Anomaly Detection
Yiqun Qiao, Jinhu Lü, Tian Wang, Kexin Liu, Baochang Zhang, Hichem Snoussi
Abstract
Industrial sensors capture critical information for intelligent manufacturing maintenance. To promote equipment upgrading and manufacturing processes, intelligent decisions, and information learning play an important role. Although deep learning methods historically obtain excellent results, there is always a tradeoff between fine-tuning existing networks or designing models from scratch for sensor data processing. In this article, we propose the multihead attention self-supervised (MAS) representation model, which is a self-supervised learning-based sensor feature extraction network. To the best of our knowledge, this is the first time a self-supervised contrastive learning method using positive samples that represent multidimensional industry sensor data is being used for anomaly detection. We review alternative data augmentation methods proposed for better-representing sensor sequence data. We use this insight to design a new structure that adapts to the temporal characteristics of the application. We apply our method to a real-world water circulation system that uses a variety of industrial sensors. The effectiveness of the proposed MAS methods is demonstrated.