Lai et al.'s Corrected Laplace Approximation improves Bayesian inference accuracy

A team of researchers from UMass Amherst and Columbia University (Jinlin Lai, Charles C. Margossian, Daniel R. Sheldon) has released a new arXiv paper proposing a corrected integrated Laplace approximation (CILA) for Bayesian inference in latent Gaussian models (LGMs). LGMs underpin many ML applications—Gaussian processes, spatial statistics, and hierarchical mixed models—but exact inference is intractable when the likelihood is non-Gaussian. The standard solution, integrated Laplace approximation (ILA), introduces approximation errors that can skew posterior estimates, especially in complex non-Gaussian settings. The authors show that by wrapping ILA with an importance sampling correction scheme, the approximate posterior converges to the correct posterior as the number of samples increases. They implement the correction using pseudo-marginalization, quasi-Monte Carlo (QMC), and randomized QMC to reduce variance, and build the system on an automatic differentiation framework that supports Hamiltonian Monte Carlo for hyperparameter inference. This makes the method practical for gradient-based sampling at scale.

In experiments across several applied models, CILA consistently reduces error compared to standard ILA, often achieving near-exact inference with far fewer samples than traditional importance sampling. The paper demonstrates that the correction is computationally feasible: the overhead is modest relative to the gains in accuracy. This is especially valuable for Bayesian practitioners who rely on LGMs but have been forced to accept ILA's biases to keep computations tractable. By providing a principled, scalable correction, CILA could become the default technique for approximate Bayesian inference in latent Gaussian models, improving reliability in fields ranging from epidemiology to reinforcement learning.