Remarks for Scaling Up a General Gaussian Process to Model Large Dataset with Sub-models
Yiming Zhang, Sayan Ghosh, Piyush Pandita, Waad Subber, Genghis Khan, Liping Wang
Abstract
Gaussian process models (GPs) have proven to be effective to approximate expensive responses (e.g. high-fidelity simulations and experiments) and widely used for industrial designs. With rapid development of computational resources and a large amount of inter-connected data, it’s becoming necessary to process large dataset efficiently for industrial designs. However, a major challenge for the standard GP is the prohibitively expensive training time with complexity O(n^3) leaving modeling data of size n=〖10〗^4 impractical. One strategy for handling large dataset is sub-modeling in which multiple smaller models are developed from sub-dataset and combined for prediction. The sub-modeling speeds up various numerical processes with GP training including Cholesky decomposition, matrix multiplications. This paper proposes intelligent sub-modeling for efficient sub-sampling and modeling from the dataset. Intelligence comes from two aspects: (1) the sub-dataset is adaptively selected to maximize the prediction accuracy of a sub-model, (2) the number of sub-models is determined based on identified stopping criterion, therefore, no need to model all the dataset. A series of techniques are integrated to enhance the intelligent sub-modeling: (1) the likelihood acquisition function for error reduction, (2) combined prediction to improve exploration of sub-sampling, (3) parameter sharing of the initial sub-model for faster training, (4) batch sampling to improve training speed and exploration of the dataset. The effect of each technique has been quantified with a test function. Comparison between a standard GP and the proposed intelligent sub-modeling is provided regarding prediction mean, uncertainty, global sensitivity and sample size. The effect of multi-objective modeling is also demonstrated with a Cantilevered Beam function.