Exploring global attention mechanism on fault detection and diagnosis for complex engineering processes
Kun Zhou, Yifan Tong, Xintong Li, Xiaoran Wei, Hao Huang, Kai Song, Xu Chen
Abstract
Considering about slow drift and complicated relationships among process variables caused by corrosion, fatigue, and so on in complex chemical engineering processes, an Industrial Process Optimization ViT (IPO-ViT) method was proposed to explore the global receptive field provided by self-attention mechanism of Vision Transformer (ViT) on fault detection and diagnosis (FDD). The applications on data sampled from both a real industrial process and the Tennessee Eastman (TE) process showed superior performance of the global attention-based method (IPO-ViT) over other typical local receptive fields deep learning methods without increasing sample and computation requirements. Moreover, results on six different variants in combing local, shallow filtering and global receptive field mechanisms unravel that the local attention explosion, the information alignment, and the expression capability are three major challenges for further improving on industrial applications of complex deep learning network structures.