New AI method measures model uncertainty, beats semantic entropy benchmarks

Researchers have introduced Directional Concentration Uncertainty (DCU), a novel framework for quantifying uncertainty in generative AI models. The method measures the geometric dispersion of model outputs using embeddings and the von Mises-Fisher distribution, requiring no task-specific heuristics. In experiments, DCU matched or exceeded the calibration performance of prior methods like semantic entropy and showed strong generalization to complex, multi-modal tasks, advancing efforts to make generative AI more robust and trustworthy.