IterCOMP's training-free prompt compression boosts multi-hop QA accuracy
New framework iteratively trims RAG contexts, lifting EM/F1 across three benchmarks.
IterCOMP, proposed by JungMin Yun and YoungBin Kim and accepted at ACL 2026, tackles a critical bottleneck in retrieval-augmented generation (RAG): multi-hop question answering often drowns models in lengthy, noisy contexts that degrade both accuracy and efficiency. Existing prompt compression methods are designed for single-turn queries, so they fail to preserve the interdependent reasoning steps needed to link evidence across multiple documents. IterCOMP solves this without any fine-tuning by embedding a reasoning-aware compression loop directly into the prompting pipeline.
Instead of compressing everything at once, IterCOMP decomposes retrieved documents into focused evidence segments, then evaluates whether the original question is answerable from the current context. If not, it generates targeted follow-up questions that guide the selection of the next piece of evidence, iteratively integrating only what's needed. This loop yields a compact, reasoning-oriented prompt that retains the answer chain while cutting token usage. Experiments across MusiQue, 2WikiMultiHopQA, and HotpotQA show substantial gains in Exact Match and F1 over baselines, with robustness that actually improves as reasoning complexity increases. For practitioners, this means cheaper, faster, and more accurate RAG systems on complex queries without model retraining.
- Training-free: IterCOMP needs no fine-tuning, working with off-the-shelf LLMs via iterative prompting.
- Outperforms baselines on MusiQue, 2WikiMultiHopQA, and HotpotQA in Exact Match and F1 while using fewer tokens.
- Uses a reasoning loop that decomposes documents, checks answerability, and generates follow-up questions to compress context adaptively.
Why It Matters
RAG developers get a plug-and-play technique to cut inference costs while improving accuracy on multi-hop reasoning tasks.