Last-layer UQ matches full-network performance, study finds

A team of researchers from the University of Melbourne and the University of Chicago has published a comprehensive study challenging the conventional wisdom about uncertainty quantification (UQ) in deep neural networks. The paper, titled "Is the Last Layer Sufficient for Uncertainty Quantification?" (arXiv:2605.30741), examines Bayesian Generalized Linear Models (GLMs) formed by linearizing DNNs. Many leading UQ methods rely on full-network linearization, which is computationally expensive. The authors instead consider last-layer linearization — a cheaper approximation often assumed to degrade performance.

Using tools from random matrix theory and conducting large-scale empirical evaluations across diverse modern ML tasks, the researchers found no meaningful improvement from full-network linearization over the last-layer approach. The paper spans 40 pages, includes 14 figures and 7 tables, and provides strong evidence that last-layer approximations deliver comparable UQ capabilities. This finding has significant implications for mission-critical AI systems like medical diagnosis or autonomous driving, where reliable uncertainty estimates are essential — and computational budgets are tight.