Zhilin Zhao's 'From Approximation to Emergence' unifies deep learning theory
A single mathematical framework covering overparameterization, transformers, scaling laws, and emergence.
Deep learning has outpaced any single mathematical explanation, but Zhilin Zhao's new monograph, 'From Approximation to Emergence: A Theory of Deep Learning,' aims to change that. Submitted to arXiv on July 1, 2026, the work presents a unified, proof-oriented framework that bridges classical foundations with modern phenomena. Rather than collecting isolated results, Zhao organizes decades of research into a coherent narrative, examining each theory through the object it controls, the assumptions that make it valid, and the gaps it leaves. The monograph targets mathematically trained practitioners and graduate students seeking a rigorous map of the field as it stands today.
Key topics range from overparameterization and robustness to transformers, in-context learning, scaling laws, interpretability, alignment, and the central question of emergence—how learned mechanisms arise from scale, data, architecture, and training. Zhao does not claim completeness; instead, the work highlights what is understood and what remains mysterious. As deep learning continues to advance, this monograph offers a crucial reference for understanding where the theory holds and where it breaks, making it essential reading for anyone serious about the foundations of modern AI.
- Unifies classical deep learning theory (approximation, optimization, generalization) with modern phenomena like overparameterization and emergence.
- Covers transformers, in-context learning, scaling laws, robustness, generative modeling, and alignment in a single mathematical framework.
- Organizes a broad literature into a coherent research narrative, explicitly noting assumptions and unexplained phenomena for each theory.
Why It Matters
Provides a foundational roadmap for researchers to understand how learned mechanisms arise from scale and data.