Point-by-point transfer learning for Bayesian optimization: An accelerated search strategy
Negareh Mahboubi, Junyao Xie, Biao Huang
Abstract
Bayesian optimization (BO) is a prominent “black-box” optimization approach. It makes sequential decisions using a Bayesian model, usually a Gaussian process, to effectively explore the search space of laborious optimization problems. However, BO faces notable challenges, particularly in constructing a reliable model for the optimization task when there are insufficient data available. To address the “cold start” problem and enhance the efficiency of BO, transfer learning appears as a powerful strategy which has gained notable attention recently. This approach aims to expedite the optimization process for a target task by utilizing knowledge accumulated from previous, related source tasks. We provide a novel point-by-point transfer learning with mixture of Gaussians for BO (PPTL-MGBO) technique to improve the speed and efficacy of the optimization process. Through evaluations on both synthetic and real-world datasets, PPTL-MGBO has demonstrated marked advancements in optimizing search efficiency, particularly when dealing with sparse or incomplete target data. • Proposes an efficient transfer learning-based Bayesian optimization method. • Integrates mixtures of Gaussians with Bayesian optimization to improve the process. • Demonstrates significant improvements on both synthetic and real-world datasets. • Validates the method’s applicability in optimization speed with limited target data.