An Adaptive Continual Learning Method for Nonstationary Industrial Time Series Prediction
Mengqing Wu, Xiaofeng Zhou, Shuai Li, Haibo Shi
Abstract
Deep learning models have gained significant attention and application in recent years to improve the accuracy and efficiency of industrial time series prediction. However, the dynamic changes in industrial processes present a key challenge for data-driven models. Specifically, the performance of deployed models deteriorates over time and fails to adapt to new operating conditions. Currently, two common update methods exist: Retraining the model using historical and new operating data, which incurs high computation and storage costs, or incrementally fine-tuning the model solely using new data, which leads to catastrophic forgetting of learned patterns. To address these issues, this article proposes an adaptive continual learning method for nonstationary industrial time series prediction. Our approach tackles the problems by hint-based network parameter learning to retain the dark knowledge from previous tasks and avoid catastrophic forgetting of accumulated knowledge. In addition, we design a soft buffer to aid memory and learning of key patterns under the current operating condition. Lastly, a time-sensitive activation function is proposed to endow the neural network with time-evolving properties, thereby enhancing the model's generalization ability. Compared with other update methods and different continual learning methods, the superiority of our method is validated on solar power generation data and real data of grinding and grading process.