Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data

A team of researchers including Saptarshi Chakraborty, Quentin Berthet, and Peter L. Bartlett has published a significant theoretical paper on arXiv that provides rigorous statistical guarantees for score-based diffusion models. The work, titled 'Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data,' addresses a major gap in understanding why models like Stable Diffusion and DALL-E perform so well in practice despite operating in high-dimensional pixel spaces. The authors prove that these models do not suffer from the classic 'curse of dimensionality' when learning real-world data distributions, such as natural images, which possess an intrinsic low-dimensional structure. Their analysis shows the expected error between the learned and true data distribution scales favorably based on this intrinsic dimension, not the ambient pixel dimension.

The key technical achievement is deriving finite-sample error bounds measured in the Wasserstein-p distance, requiring only a finite-moment assumption on the data and holding for all p ≥ 1. The convergence rate is shown to be ~O(n^{-1/d*_{p,q}}), where d*_{p,q} is a newly defined (p,q)-Wasserstein dimension that captures the data's geometric complexity. This result conceptually bridges the analysis of diffusion models with that of GANs and optimal transport theory. For practitioners, it provides a mathematical foundation confirming that diffusion models are not just empirically successful but are provably efficient learners of complex, real-world data, which helps justify their architecture choices and guides future model development toward leveraging intrinsic data geometry.