New math proof shows limits of neural networks for solving PDEs
Deep Ritz method faces fundamental regularity constraints in Barron spaces.
A new mathematical paper by Stephan Wojtowytsch tackles a fundamental question at the intersection of deep learning and partial differential equations (PDEs): Can neural networks with bounded weights reliably approximate smooth solutions to elliptic equations? The answer is nuanced. The work focuses on Barron space, a function class tailored to wide ReLU networks with a single hidden layer. Wojtowytsch proves that harmonic functions with Dirichlet boundary data in Barron space are generally neither Lipschitz continuous nor in the Sobolev class H². This rules out not only Barron space regularity but also regularity in deeper ReLU networks with bounded coefficients, revealing a structural limitation of these models for PDE solving.
On the positive side, the paper shows that these same harmonic functions can be approximated to accuracy ~ε by Barron functions of low norm ~|log ε| in various Lebesgue and Sobolev norms (with at most two derivatives). This approximation holds on simple domains: half-spaces in any dimension and rectangular domains in two dimensions. As a direct application, Wojtowytsch derives a priori error estimates for the Deep Ritz method, a popular neural-network-based approach for solving variational PDEs. These estimates provide rigorous bounds on how well the neural solver can approximate the true solution, given the regularity constraints identified.
- Harmonic functions in Barron space are generally not Lipschitz continuous or in Sobolev class H², limiting neural network expressivity for PDEs.
- Approximation to accuracy ε is possible with Barron norm scaling like |log ε| on half-spaces and 2D rectangles.
- Results provide rigorous a priori error bounds for the Deep Ritz method, a neural PDE solver.
Why It Matters
Forces AI-driven scientific computing to acknowledge fundamental mathematical limits of neural networks for PDE approximation.