ILDM: Hybrid latent diffusion learns manifold geometry, cuts FID scores

Diffusion models have become state-of-the-art for high-dimensional data synthesis, but they typically ignore intrinsic geometric structure and require large datasets. Latent diffusion models (LDMs) reduce dimensionality by learning a Euclidean latent space, yet this fails on data that lies on a curved manifold—especially when data is scarce. The Intrinsic Hybrid Latent Diffusion Model (ILDM), proposed by Yizhu Wang, Mu Niu, and Xiaochen Yang, tackles this by interpreting the latent space as a chart of an unknown Riemannian manifold, with geometry and uncertainty quantified through a probabilistic decoder.

ILDM's forward process is a hybrid diffusion that dynamically switches between Riemannian and Euclidean dynamics based on local uncertainty. The Riemannian component uses a probabilistic metric tensor derived from the decoder, and the authors introduce an approximate denoising score matching method for this hybrid setting, enabling a backward process driven by hybrid Langevin dynamics. In experiments on COIL-100, MNIST, and cardiac MRI, ILDM significantly improves generation quality, achieving lower FID and LPIPS scores than standard diffusion and latent diffusion baselines. This makes ILDM a promising direction for generative modeling on complex, low-dimensional structures like medical imaging and object recognition.