Enhancing Training of Physics-Informed Neural Networks Using Domain Decomposition–Based Preconditioning Strategies
Alena Kopaničáková, Hardik Kothari, George Em Karniadakis, Rolf Krause
Abstract
.We propose to enhance the training of physics-informed neural networks. To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain decomposition framework, where the parameters of the network are decomposed in a layerwise manner. Through a series of numerical experiments, we demonstrate that both additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer while providing more accurate solutions of the underlying PDEs. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.Keywordsscientific machine learningnonlinear preconditioningSchwarz methodsdomain decompositionMSC codes90C3090C2690C0665M5568T07