New framework uses LLM uncertainty signals for smarter RAG decisions
Detects when to retrieve and when to reason – in a single forward pass.
A new paper from University of Glasgow researchers introduces an interpretable uncertainty framework for adaptive retrieval and reasoning in question answering (QA). The method, detailed in arXiv:2607.07380, addresses a key limitation of retrieval-augmented generation (RAG): when to retrieve and when to rely on the LLM's own knowledge. Current approaches use opaque policies or multi-step prompting, which are inefficient and non-transparent.
The proposed solution decomposes uncertainty into two types: knowledge insufficiency (the model doesn't know) and knowledge ambiguity/conflict (the model has conflicting information). By extracting signals from the LLM's hidden states in a single forward pass, the framework decides whether to trigger RAG (for insufficient knowledge) or apply additional reasoning (for high ambiguity). This makes the system's behavior interpretable and computationally efficient, offering a practical alternative for building user-facing QA tools.
- Decomposes uncertainty into knowledge insufficiency vs. ambiguity/conflict using LLM hidden states
- Triggers RAG only when needed, and extra reasoning when ambiguity is high – all in one forward pass
- Provides transparent decision-making compared to opaque retrieval policies or multi-step prompting
Why It Matters
Makes QA systems more efficient and trustworthy by revealing when and why they retrieve external information.