KF-PLS: Optimizing Kernel Partial Least-Squares (K-PLS) with Kernel Flows
Zina-Sabrina Duma, Jouni Susiluoto, Otto Lamminpää, Tuomas Sihvonen, Сату-Пиа Рейникайнен, Heikki Haario
Abstract
Partial Least-Squares (PLS) regression is a widely used tool in chemometrics for performing multivariate regression. As PLS has a limited capacity of modelling non-linear relations between the predictor variables and the response, Kernel PLS (K-PLS) has been introduced for modelling non-linear predictor-response relations. Most available studies use fixed kernel parameters, reducing the performance potential of the method. Only a few studies have been conducted on optimizing the kernel parameters for K-PLS. In this article, we propose a methodology for the kernel function optimization based on Kernel Flows (KF), a technique developed for Gaussian Process Regression (GPR). The results are illustrated with four case studies. The case studies represent both numerical examples and real data used in classification and regression tasks. K-PLS optimized with KF, called KF-PLS in this study, is shown to yield good results in all illustrated scenarios, outperforming literature results and other non-linear regression methodologies. In the present study, KF-PLS has been compared to convolutional neural networks (CNN), random trees , ensemble methods, support vector machines (SVM), and GPR, and it has proved to perform very well.