New Proof Shows Diffusion Models Are Statistically Optimal for Low-Dim Data

Score-based diffusion models have excelled empirically on high-dimensional data like images, but theoretical understanding of their statistical efficiency lagged behind. Prior analyses often required strong assumptions—such as uniformly bounded densities or globally smooth score functions—that don't hold for real-world distributions with low-dimensional and multi-modal structures. In a new paper accepted to ICML 2026, Jingda Wu and Changxiao Cai tackle this gap by studying diffusion models learning distributions that lie on a union of low-dimensional subspaces. They assume only that within each subspace the data is subgaussian, a much weaker condition than prior work.

Their main result: diffusion models need at most ~O(ε^{-k∨2}) samples to achieve ε error in 1-Wasserstein distance, where k is the intrinsic dimension. This rate is statistically near-optimal and depends only on the intrinsic dimension, not the ambient dimension—so diffusion models automatically adapt to low-dimensional structure. The proof works for a broad class of multi-modal distributions without requiring smoothness, bounded densities, or log-concavity. These findings offer a rigorous theoretical foundation for why diffusion models succeed on complex data like images and audio, where intrinsic dimension is often much lower than pixel or sample counts. The work also suggests that further architectural improvements could leverage this intrinsic adaptation.