An Improved Deep Kernel Partial Least Squares and Its Application to Industrial Data Modeling
Yongxuan Chen, Dianhui Wang
Abstract
Deep kernel partial least squares (DeKPLS) is a recently proposed deep learning model for nonlinear regression analysis. By combining the principle of deep learning and kernel partial least squares (KPLS), DeKPLS can extract intrinsic data features in a more concise way. However, the original DeKPLS method, which simply stacks multiple KPLS modules, may cause the accumulation of information loss due to the dimension-reduction operations of KPLS. To alleviate this problem, this article proposes a novel nonconnectionist deep learning model based on DeKPLS, which is referred to as reinforced DeKPLS (ReKPLS). Inspired by the concept of residual networks, the unprocessed inputs from the previous layer are identity-mapped to the next layer of ReKPLS, which is helpful for reserving useful regression information. Through this simple modification, ReKPLS can improve the prediction performance with nearly no extra training costs compared to DeKPLS. Two real-world industrial cases are used to illustrate the effectiveness, including prediction accuracy and modeling efficiency, of our proposed method.