Partition Function Estimation under Bounded f-Divergence

Researchers Adam Block and Abhishek Shetty have published a significant theoretical paper, 'Partition Function Estimation under Bounded f-Divergence,' that provides a foundational, minimal-assumption framework for a core problem in machine learning and statistics. The work tackles the challenge of estimating partition functions (or normalizing constants), which is essential for probabilistic modeling, Bayesian inference, and evaluating generative models. Unlike previous approaches that relied on specific structural or geometric assumptions, the new theory offers a general characterization based solely on the relationship between a proposal distribution and a target distribution, quantified through f-divergences like KL divergence. This shift moves the field toward a more universal understanding of the problem's inherent statistical difficulty.

The paper's key innovation is the introduction of the 'integrated coverage profile,' a functional that measures how much probability mass from the target distribution lies in regions where the density ratio is large. The authors prove this profile tightly characterizes the sample complexity for achieving a multiplicative approximation of the partition function, and they provide matching information-theoretic lower bounds to establish optimality. As major applications, the framework yields improved finite-sample guarantees for importance sampling and self-normalized importance sampling, critical algorithms for approximate inference. Furthermore, it demonstrates a strict separation in complexity between the tasks of approximate sampling and approximate counting under the same divergence constraints, a finding with implications for the theoretical limits of algorithms in fields like statistical physics and combinatorial optimization. The work unifies and generalizes prior analyses, creating a cohesive theory that connects importance sampling, rejection sampling, and heavy-tailed mean estimation.