Why PINN's residual loss saves gradient-based PDE inversion from failure

Yan Yang's paper, accepted at the AI4Physics Workshop at ICML 2026, tackles a fundamental problem in physics-informed machine learning: why does gradient-based inversion of reaction-diffusion systems often fail when using the most direct route – backpropagating through the PDE itself? The study uses the Gray-Scott reaction-diffusion system as a testbed, unrolling its simulation and backpropagating a steady-state loss to recover parameters without any surrogate or neural-network augmentation. The result: optimization consistently fails to converge. By plotting the loss landscape, Yang identifies the root cause – flat plateaus with zero gradient signal, sharply bounded by cliffs that align with bifurcation boundaries of the PDE. This pathological structure persists across loss functions and gradient routing methods, revealing a fundamental geometric obstruction.

Reading this minimal setup as an ablation of a Physics-Informed Neural Network (PINN), Yang systematically disentangles the roles of each PINN component. With the neural network fixed, the residual loss is quadratic in the PDE parameters, producing a smooth, convex landscape that avoids the plateau pathology entirely. Why? Because the residual loss implicitly encodes the full PDE dynamics across all initial conditions, a property that direct simulation backpropagation lacks. The neural network, meanwhile, cannot repair an ill-posed parameter subspace – its role is limited to fitting the observed data, not fixing geometry. This division of labor, previously not made explicit, carries concrete design implications: PINN-style methods should prioritize the residual loss component and recognize that adding neural network capacity alone won't fix fundamental parameter recovery issues. The work also offers a broader heuristic on when adding dimensions actually helps.