A Faster Dynamic Feature Extractor and Its Application to Industrial Quality Prediction
Bocun He, Qingzhi Zhang, Xinmin Zhang
Abstract
The unsupervised dynamic models have been applied to various tasks in the process industry due to their excellent ability to represent the process dynamics. The recurrent-network-based dynamic feature extractor is a typical unsupervised dynamic model which extracts the dynamic data features using a recurrent encoder network. However, the recurrent-network-based dynamic feature extractor has low computational efficiency due to its recurrent nature, which prevents the model from being used for large-scale data sets. To improve computational efficiency, a new dynamic feature extractor called TempoATTNE-DFE is proposed in this work. In TempoATTNE-DFE, a new encoder structure is developed, which can be implemented in parallel for data sequences. Meanwhile, a kind of attention mechanism is proposed to extract the dynamic features within the input sequence. The proposed TempoATTNE-DFE can achieve higher computational efficiency in offline training and online inference. To evaluate the effectiveness of TempoATTNE-DFE, it is applied to the quality prediction task and validated with a numerical example and two real-world industrial processes. The application results demonstrate that TempoATTNE-DFE can achieve better prediction performance compared to other state-of-the-art methods. In addition, compared with the recurrent-network-based dynamic feature extractor, TempoATTNE-DFE gains <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$1.29\times$</tex-math></inline-formula> speedup in training and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$2.45\times$</tex-math></inline-formula> speedup in inference on the blast furnace data set.