ELADO benchmark exposes hidden failures in neural PDE operators

Neural operators have become a popular tool for learning solution mappings of partial differential equations, but their reliability in real-world scenarios remains unclear. A new paper on arXiv introduces ELADO (Elliptic PDE Assessment Datasets for Operator Learning), a systematic benchmark suite designed to expose and quantify failure modes of neural operator architectures. Created by Frank Ehebrecht, Toni Scharle, and Martin Atzmueller, the datasets are built around Poisson's equation and the Helmholtz equation with non-constant coefficients. Unlike existing benchmarks that focus on average-case performance, ELADO constructs controlled data-generating processes to isolate distinct sources of difficulty, revealing hidden weaknesses that standard metrics overlook.

The benchmark specifically targets five failure modes: (1) heavy-tailed solution distributions arising from light-tailed coefficient fields, (2) spectral distribution shift of input data, (3) heavy-tailed distributions in the frequency domain of solutions, (4) input sensitivity of learned operators (quantified via empirical local Lipschitz analysis), and (5) the effect of input signal complexity on prediction accuracy under controlled amplitude normalization. The authors evaluated several neural operator architectures across all datasets and found that heavy-tailed targets, spectral shift, and input sensitivity each cause substantial degradation in prediction accuracy that is often obscured by the commonly used mean relative L2 error. This work underscores the importance of stress-testing neural operators beyond standard benchmarks, especially for applications in engineering and scientific computing where PDE solution accuracy is critical.