New method reveals deep learning's hidden uncertainty sources

Researchers Pierre Nodet and Thomas George have introduced a method to dissect and quantify uncertainty in deep learning predictions, addressing a critical gap in AI reliability. Their work, published on arXiv, adapts classical statistical estimators—specifically homo- and hetero-scedastic linearized estimators—to modern neural networks, enabling practitioners to distinguish between aleatoric uncertainty (stemming from sparse or noisy data) and epistemic uncertainty (arising from model limitations).

The breakthrough leverages approximate Fisher Information Matrices, a technique that allows the method to scale effectively with actual deep learning architectures, including those used in production. Experimental results demonstrate that individual test points are differentially impacted by these two sources of uncertainty, highlighting how the approach can improve the robustness of AI systems in high-stakes applications like healthcare or autonomous driving. By providing clearer insights into prediction confidence, the method could help engineers build more reliable and interpretable AI models.