New local geometric framework improves generative AI score estimation
Replaces global averaging with local heat-ball geometry to reduce sampling errors in low-density regions.
Score-based generative models rely on estimating the score function of a forward diffusion. Current methods like denoising score matching (DSM) or global Fokker-Planck residuals suffer from inflated error in low-density regions. Bai et al. address this by developing a local Fokker-Planck geometry framework. Their first contribution is a time change to the cumulative-variance coordinate, reducing the variable-coefficient FP equation to an inhomogeneous heat equation. This enables exact local mean-value representations for the score, log-density, and entropy density via extended heat-ball monotonicity methods. Local well-posedness is guaranteed under an explicit dimension-dependent drift budget.
For high-dimensional Monte Carlo evaluation, they introduce the κ-measure with a factorized sampler yielding unit per-sample weight and χ^2_2 radial concentration. The r→0 limit of the heat-ball residual recovers the pointwise Fokker-Planck residual, showing local FP is a one-parameter generalization of global methods. The framework is validated on 2D structured data and 256-dimensional MNIST, demonstrating improved accuracy. This work offers a more principled way to handle nonlinear drifts and low-density regions, potentially enabling better generative sampling and faster convergence in high-dimensional settings.
- Introduces heat-ball mean-value representations for exact local score estimation, replacing global averaging.
- κ-measure and factorized sampler achieve unit per-sample weight with χ^2_2 radial concentration for high-dimensional integration.
- Validated on 256-dimensional MNIST, showing reduced errors in low-density regions vs. denoising score matching and global FP residuals.
Why It Matters
Improves accuracy of generative models in data-sparse regions, enabling more reliable sampling for high-dimensional AI applications.