New Math Theory Could Make AI Safer and Smarter

This writeup analyzes the special case where the true distribution is regular for the statistical model, which covers classical Bayesian statistics. It examines the regular posterior distribution and the universal structure that is its asymptotic behavior, a theorem akin to the central limit theorem called the Bayesian central limit theorem, or Bernstein–von Mises theorem. It also looks at expansions of the observables, introduces point estimators, and covers conventional asymptotic theory and the basic Bayesian treatment.

Notably, the writeup has already forgone realizability, a major assumption that need not hold in models of interest such as the neural networks used in practice. The first goal is to estimate the partition function, which is a Laplace integral. Starting with a simpler example, the article builds the solution step by step: centering the integral, doing a Taylor expansion of the exponent with the Lagrange remainder, splitting the integral into a neighborhood where most of the mass is concentrated and a remainder, and showing the second integral is negligible compared to the first. The control lies in choosing the right neighborhood and in bounding the third-order Taylor term so it is less than a given factor of the second-order term. The article notes it is recommended to read the writeups on primer probability and the Laplace method first, and that the main references continue to be the Watanabe books.