When More Reformulations Hurt: Avoiding Drift using Ranker Feedback
ReformIR boosts recall while minimizing query drift and computational costs.
In the recent study, V Venktesh, Mandeep Rathee, and Avishek Anand present ReformIR, a revolutionary framework designed to enhance the efficiency of query retrieval systems. The key challenge in modern retrieval pipelines is balancing recall and the risk of query drift, which arises from generating excessive reformulated queries. ReformIR addresses this by implementing a budget-aware approach that treats reformulations as vital features, using a strong reranker as a teacher to refine relevance estimation dynamically. This innovative method allows for a significant increase in recall without incurring the computational costs typically associated with exhaustive reranking.
Extensive experiments conducted on MSMARCO passage corpora and TREC benchmarks demonstrate that ReformIR consistently outperforms existing reformulation techniques, particularly in scenarios with multiple reformulations where traditional methods experience quality degradation. The findings suggest a paradigm shift in retrieval system design; rather than relying solely on large language models for reranking, their capabilities can be more effectively utilized during the reformulation phase through feedback-driven optimization. This could lead to more efficient retrieval systems that provide higher quality results while managing computational resources effectively.
- ReformIR framework optimizes query reformulations, enhancing recall while controlling costs.
- Employs adaptive selection of reformulations and documents under fixed inference budgets.
- Outperforms existing methods in benchmark tests, reducing drift by 30%.
Why It Matters
Enhances retrieval system performance while managing computational resources effectively.