'Learning Mechanics' Paper Proposes Physics-Like Theory for Deep Learning

A new paper, 'There Will Be a Scientific Theory of Deep Learning', proposes that a physics-like mathematical framework called 'learning mechanics' is emerging to explain how neural networks learn. Borrowing from classical mechanics, statistical mechanics, and quantum mechanics, this approach focuses on the dynamics of training, coarse-grained aggregate statistics, and falsifiable quantitative predictions. The authors argue that deep learning theory has historically lagged behind empirical results, but this shift toward a mechanics-inspired perspective could change that. The paper highlights recent NeurIPS workshops on dynamical systems in ML as evidence of this growing trend.

Crucially, the authors position learning mechanics as complementary to mechanistic interpretability—analogous to the relationship between physics and biology. While mechanistic interpretability tries to reverse-engineer individual components (like biological anatomy), learning mechanics seeks overarching mathematical laws (like physics). The author of this summary, whose own research aligns with this view, cautions that different levels of description will always coexist. The framework is not meant to replace all approaches but to offer a rigorous, first-principles foundation for understanding learning at scale.