Variational Inference Over Graph: Knowledge Representation for Deep Process Data Analytics
Zhichao Chen, Zhihuan Song, Zhiqiang Ge
Abstract
With the advent of the industrial Big Data era, accurate estimation of product quality and monitoring of working conditions from historical data have become crucial in the process industry. However, the majority of data-driven approaches predominantly rely on observational data, overlooking the valuable empirical knowledge derived from experience or underlying mechanisms. In order to leverage this knowledge, researchers employ various graph neural network-based methods which introduce connections among process variables for feature extraction. Nevertheless, it is imperative to recognize that process knowledge undergoes changes due to internal or external concept drift. To address this challenge, we propose a novel deep learning module called “variational inference over graph” to effectively harness shifting knowledge. Building upon the self-attention mechanism, we design a probabilistic self-attention mechanism for encoding and reconciling prior knowledge. Instead of directly encoding the prior knowledge through graph neural network edges, we incorporate it as regularization term within the variational inference framework that accounts for knowledge shift. Furthermore, we introduce reparameterization estimator to control the variance resulting from knowledge uncertainty. To showcase the capability of our proposed method, we conduct various experiments on quality prediction task in real industrial processes.