Engression gets rigorous error analysis with deep neural networks

Engression, introduced by Shen and Meinshausen in 2024, is a statistical learning approach that models conditional distributions by fitting a generative function Y = f(X, ε) using the energy score, a strictly proper scoring rule. In this new arXiv paper, Chen, Guo, and Shen provide the first in-depth theoretical error analysis of engression when implemented with deep neural networks. The authors decompose the excess risk into three transparent components: approximation error (how well the network class captures the target), stochastic error (finite-sample estimation variance), and Monte Carlo error (random noise from sampling ε). This decomposition allows them to derive explicit convergence rates for engression under the assumption that the conditional generator has a compositional smoothness structure—a common and practical condition for deep learning theory.

The paper spans 37 pages and includes a single figure, but its significance lies in closing a key gap in the literature. Prior work focused on engression's empirical performance, but theory lagged behind. By proving that engression achieves fast convergence rates when implemented with neural networks, the authors offer practitioners a rigorous guarantee that engression can reliably estimate conditional distributions, not just conditional means. This positions engression as a principled alternative to quantile regression or GANs for uncertainty-aware prediction. The decomposition also suggests practical diagnostic tools: users can isolate Monte Carlo error by increasing ε samples or reduce stochastic error via larger datasets. For the ML community, this paper strengthens the theoretical foundations of energy-based generative modeling and opens avenues for extending similar analyses to other scoring-rule-based learning paradigms.