Score Accuracy Doesn't Guarantee Stable Diffusion Sampling, New Paper Shows

Score matching optimizes average error under forward diffusion marginals, but a discretized reverse-time sampler evaluates the learned score along its own trajectory. Yiwei Zhou's paper shows that small forward-marginal error does not guarantee numerical stability. The author constructs a smooth score field with arbitrarily small L² forward error, yet its Euler-Maruyama discretizations converge in probability while every positive moment diverges. This means weak convergence can hold even though every Wasserstein distance W_p (p≥1) diverges. The failure occurs within fixed finite neural architectures. For compactly supported data, a simple fix exists: projecting the learned denoiser onto a bounded closed convex set containing the support preserves pointwise accuracy, gives grid-uniform moment bounds, and yields Wasserstein convergence under mild local regularity.

Experiments with a small fixed DiT-style network demonstrate large growth along rare numerical trajectories and its suppression by denoiser projection, while overall trajectory errors remain small. This research highlights a critical blind spot in diffusion model evaluation—focusing only on score accuracy can miss severe instability in sampling. The positive result offers a practical mitigation for models trained on bounded data, but extensions to unbounded domains remain open. For practitioners deploying diffusion samplers (e.g., image, video, audio generation), this work underscores the need for numerical stability checks beyond standard benchmark metrics.