Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification

Researchers Courtney Franzen and Farhad Pourkamali-Anaraki have developed a new ensemble-based technique that addresses a critical weakness in modern neural networks: their inability to provide reliable predictive uncertainty estimates. Standard classifiers trained with cross-entropy loss achieve high accuracy but lack inherent uncertainty quantification, while existing solutions like Evidential Deep Learning (EDL) are notoriously fragile—highly sensitive to loss formulation, prior regularization, and activation functions. The new method sidesteps these design challenges by applying a method of moments estimator to ensembles of softmax outputs from multiple training runs, optionally refined with maximum-likelihood estimation. This ensemble-based construction directly produces explicit Dirichlet predictive distributions, decoupling robust uncertainty estimation from the unstable training process of single-model evidential approaches.

By aggregating predictions across ensembles, the method significantly mitigates the substantial variability of softmax scores for the true class that plagues independent training runs. This increased stability in uncertainty estimation has direct, practical implications for downstream applications that rely on AI confidence. The paper demonstrates across multiple datasets that the ensemble-derived Dirichlet estimates lead to stronger performance in uncertainty-guided tasks, particularly in selective classification—where a model must decide when to make a prediction or abstain based on its confidence. This represents a meaningful step toward more reliable and trustworthy AI systems in high-stakes domains where understanding what the model doesn't know is as important as what it does.