TASR: Training-Free Stopping Rule Cuts RAG Costs by 37%
New one-line predicate saves retrieval calls without losing accuracy.
Iterative retrieval-augmented generation (RAG) agents often waste compute by continuing to retrieve after the model has already converged on a correct answer. TASR solves this with a training-free, auditable stopping rule: it fires when the model repeats its previous-round normalized answer and the isotonically calibrated logit margin exceeds 0.25. No classifier or value head is learned, making it immediately transferable across models, retrievers, and corpora.
Evaluated on 24 (model, retriever, corpus) configurations, TASR retains 94.8% of fixed-k=5's macro F1 while using only 62.6% of its calls, and exceeds fixed-k=3 by +3.42 F1. The rule was selected from 381 candidates and Pareto-dominates all alternatives. A signal-quality analysis reveals that RLHF-tuned models collapse verbalized confidence (96.5% of values = 5, entropy 0.182 nats), while the logit margin achieves 44x better class-conditional separation, grounding the design in a measurable pathology.
- TASR uses a one-line predicate (logit margin > 0.25 + answer repetition) to stop retrieval adaptively.
- Saves 37.4% of retrieval calls while retaining 94.8% of F1 score across 3 models and 2 datasets.
- Outperforms fixed-k=3 by +3.42 F1 with zero training requirements—tested on 24 configurations.
Why It Matters
Reduces RAG compute costs by over a third without sacrificing accuracy, enabling cheaper, faster AI agents.