Stochastic consensus dynamics for nonconvex optimization on the Stiefel manifold: Mean-field limit and convergence
Seung‐Yeal Ha, Myeongju Kang, Dohyun Kim, Jeongho Kim, Insoon Yang
Abstract
We study a consensus-based method for minimizing a nonconvex function over the Stiefel manifold. The consensus dynamics consists of stochastic differential equations for interacting particle system, whose trajectory is guaranteed to stay on the Stiefel manifold. For the proposed model, we prove the mean-field limit of the stochastic system toward a nonlinear Fokker–Planck equation on the Stiefel manifold. Moreover, we provide a sufficient condition on the parameter and the initial data, so that the solution to the Fokker–Planck equation is asymptotically concentrated on the point near a global optimizer. To implement our consensus-based optimization (CBO) algorithm, we provide two algorithms; one is improved from the algorithm suggested in our previous work, and the other is based on an entirely different approach, namely the Cayley transformation. We validate the CBO algorithms on the various test problems on the Stiefel manifold.