Exponential ReLU Neural Network Approximation Rates for Point and Edge Singularities
Carlo Marcati, Joost A. A. Opschoor, Philipp Petersen, Christoph Schwab
Abstract
Abstract In certain polytopal domains $$\varOmega $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>Ω</mml:mi> </mml:math> , in space dimension $$d=2,3$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>d</mml:mi> <mml:mo>=</mml:mo> <mml:mn>2</mml:mn> <mml:mo>,</mml:mo> <mml:mn>3</mml:mn> </mml:mrow> </mml:math> , we prove exponential expressivity with stable ReLU Neural Networks (ReLU NNs) in $$H^1(\varOmega )$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:msup> <mml:mi>H</mml:mi> <mml:mn>1</mml:mn> </mml:msup> <mml:mrow> <mml:mo>(</mml:mo> <mml:mi>Ω</mml:mi> <mml:mo>)</mml:mo> </mml:mrow> </mml:mrow> </mml:math> for weighted analytic function classes. These classes comprise in particular solution sets of source and eigenvalue problems for elliptic PDEs with analytic data. Functions in these classes are locally analytic on open subdomains $$D\subset \varOmega $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>D</mml:mi> <mml:mo>⊂</mml:mo> <mml:mi>Ω</mml:mi> </mml:mrow> </mml:math> , but may exhibit isolated point singularities in the interior of $$\varOmega $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>Ω</mml:mi> </mml:math> or corner and edge singularities at the boundary $$\partial \varOmega $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>∂</mml:mi> <mml:mi>Ω</mml:mi> </mml:mrow> </mml:math> . The exponential approximation rates are shown to hold in space dimension $$d = 2$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>d</mml:mi> <mml:mo>=</mml:mo> <mml:mn>2</mml:mn> </mml:mrow> </mml:math> on Lipschitz polygons with straight sides, and in space dimension $$d=3$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>d</mml:mi> <mml:mo>=</mml:mo> <mml:mn>3</mml:mn> </mml:mrow> </mml:math> on Fichera-type polyhedral domains with plane faces. The constructive proofs indicate that NN depth and size increase poly-logarithmically with respect to the target NN approximation accuracy $$\varepsilon >0$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>ε</mml:mi> <mml:mo>></mml:mo> <mml:mn>0</mml:mn> </mml:mrow> </mml:math> in $$H^1(\varOmega )$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:msup> <mml:mi>H</mml:mi> <mml:mn>1</mml:mn> </mml:msup> <mml:mrow> <mml:mo>(</mml:mo> <mml:mi>Ω</mml:mi> <mml:mo>)</mml:mo> </mml:mrow> </mml:mrow> </mml:math> . The results cover solution sets of linear, second-order elliptic PDEs with analytic data and certain nonlinear elliptic eigenvalue problems with analytic nonlinearities and singular, weighted analytic potentials as arise in electron structure models. Here, the functions correspond to electron densities that exhibit isolated point singularities at the nuclei.