Controllable Evidence Selection in Retrieval-Augmented Question Answering via Deterministic Utility Gating
A new paper introduces fixed scoring rules to filter retrieved text, requiring each piece to explicitly state the needed fact.
A new research paper by Victor P. Unda tackles a core weakness in modern Retrieval-Augmented Generation (RAG) systems. While these AI question-answering tools are great at finding topically similar text, they often struggle to distinguish between text that is merely similar and text that can actually serve as valid evidence. This leads to answers based on redundant, incomplete, or contextually mismatched information.
The proposed solution is a deterministic framework called Controllable Evidence Selection via Deterministic Utility Gating. It introduces two fixed, rule-based scoring procedures: Meaning-Utility Estimation (MUE) and Diversity-Utility Estimation (DUE). These procedures evaluate each retrieved sentence or record independently against explicit signals like semantic relatedness, term coverage, and conceptual distinctiveness. Crucially, a unit is only accepted as evidence if it explicitly and independently states the fact, rule, or condition required by the user's question. If no single unit meets the bar, the system returns no answer rather than fabricating one from imperfect parts.
This approach requires no model training or fine-tuning, operating on clear, auditable rules. The result is a system that establishes a strict boundary between relevant text and usable evidence, producing compact and verifiable evidence sets. This moves RAG systems from being probabilistic 'black boxes' toward more reliable and transparent reasoning tools, especially for high-stakes applications where answer provenance is critical.
- Introduces deterministic utility gating with MUE and DUE scoring to filter RAG-retrieved text.
- Requires each evidence unit to explicitly state the needed fact independently, rejecting merged or expanded context.
- Operates without training, creating auditable evidence trails and returning no answer if standards aren't met.
Why It Matters
This makes AI answers more reliable and auditable for professionals in legal, medical, and financial fields where evidence provenance is critical.