Singular Learning Theory Part 2: Formalizing Realizability, Regularity, and Generating Functions
How mathematical assumptions shape our understanding of neural network loss landscapes.
Agastya Agrawal's second installment in the Singular Learning Theory series on LessWrong dives deep into the formal mathematical framework needed to analyze Bayesian learning in non-regular models (like neural networks). It begins by defining the core concepts: realizability (whether the model can exactly represent the true distribution) and regularity (conditions for a simple parabolic loss landscape around the optimum). The post emphasizes that while regularity holds in classical statistics, neural networks typically violate it due to degenerate minima, making the theory essential for practical AI.
To unify the various observables (free energy, training loss, WAIC), Agrawal introduces the cumulant generating function of the Bayesian predictive distribution. He outlines a step-by-step procedure starting from the formal relation between true distribution and model, through normalized observables, to deriving asymptotic behaviors. The post references Watanabe's original work but adapts the explanation to clarify relationships between terms. It also provides a companion notebook with examples and exercises. This mathematical groundwork is crucial for understanding why models like large language models exhibit non-standard learning curves and validation behavior.
- Defines realizability: existence of a parameter that makes the model equal to the true distribution almost surely.
- Defines regularity: the set of optimal parameters is a single point in the interior with a positive definite Hessian – violated in neural networks.
- Uses cumulant generating functions to connect Bayesian observables (free energy, losses, WAIC) for asymptotic analysis.
Why It Matters
Provides theoretical tools to understand why neural networks behave differently from classical statistical models.