New SiLD framework proves diffusion models beat curse of dimensionality

A team of researchers from Singapore, Japan, and Australia has introduced Score-induced Latent Diffusion (SiLD), a novel theoretical and practical framework that explains why diffusion models can efficiently learn high-dimensional data that actually lies on low-dimensional manifolds. The paper, published on arXiv, identifies a "collapse-and-refine" mechanism driven by the geometry of the score function itself. At small noise scales, the diverging singularity of the score forces a rapid dimensional collapse of the denoising map onto the data manifold. At moderate noise scales, training refines the intrinsic density on that learned manifold. This replaces the heuristic KL regularization used in VAE-based latent diffusion models (like Stable Diffusion) with a single denoising score matching objective, making the entire process provably dependent on the intrinsic dimension rather than the ambient dimension.

SiLD was tested on several benchmarks including Stacked MNIST, CelebA variants, and molecular generation tasks. Results show it matches or outperforms VAE-based LDMs in generation quality (FID scores) and consistently improves reconstruction accuracy. The work provides the first theoretical proof that diffusion models can escape the curse of dimensionality under the manifold hypothesis—a long-standing open question in generative modeling. By unifying manifold learning and density estimation into one objective, SiLD opens the door to more efficient and theoretically grounded diffusion models for high-dimensional applications like 3D molecule generation and medical imaging.