FUSE framework quantifies VLM uncertainty with Bayesian fusion
New method combines data ambiguity and model diversity for SOTA calibration.
Researchers Harry Zhang and Luca Carlone introduce FUSE, a novel Bayesian framework designed to quantify uncertainty in vision-language models (VLMs) by fusing two complementary sources: aleatoric uncertainty (arising from ambiguity in input data) and epistemic uncertainty (stemming from model-level variability in semantic responses). Unlike prior approaches that treat these independently, FUSE analytically combines them using a Bayesian mechanism to produce a single, calibrated scalar that indicates how confident the VLM is in its output.
In practical terms, this means VLMs—now widely used in robotics, autonomous navigation, and visual question answering—can now output a reliable confidence score alongside their predictions. The framework achieves state-of-the-art uncertainty calibration, enabling downstream systems to better decide when to trust a model’s prediction or when to flag ambiguous cases for human review. FUSE is particularly critical for safety-critical applications where incorrect VLM outputs could lead to costly failures.
- Fuses aleatoric (data ambiguity) and epistemic (model diversity) uncertainty into a single scalar measure.
- Achieves state-of-the-art calibration on VLM benchmarks, outperforming existing baselines.
- Enables reliable prediction of output correctness for safety-critical downstream applications like robotics.
Why It Matters
Better VLM uncertainty quantification means safer robotic and autonomous systems that know when to ask for help.