Bridge Evidence Study: Static Retrieval Fails to Predict Agentic Search Utility

A new preprint from Debayan Mukhopadhyay, Utshab Kumar Ghosh, and Shubham Chatterjee challenges the foundation of how we evaluate retrieval systems in multi-step agentic search. The paper introduces Counterfactual Trajectory Utility (CTU)—a measure that deletes a document mid-trajectory and re-runs the agent from that point to observe the causal impact on final answer quality, next query retrieval quality, and turn count. Comparing CTU against Static RAG Utility (SRU) across 23,322 document observations from a ReAct agent on HotpotQA, the authors found a Spearman correlation of just -0.026—effectively statistical independence. Roughly a third of the documents read by the agent are 'bridge documents': they are causally load-bearing for the search trajectory but look useless to a static reader. Even when using BM25 and cross-encoders as the static proxy, 27.2% of documents fall into this bridge cell.

The mechanism behind bridge documents is pinned down using Observable Entity Relevance (OER). Entities that discriminate relevant from non-relevant candidates appear in the agent's next query 4.02 times more often than entities found only in non-relevant documents (6.1% vs 1.5%, n=227,139). A bridge document earns its value by providing a discriminative entity that redirects the search—not by directly answering the current question. This finding has serious implications for RAG systems and agentic tool use: optimizing for static relevance (e.g., nDCG, MRR) does not guarantee causal usefulness in multi-turn settings. The authors argue that evaluation frameworks for retrieval-augmented agents must incorporate counterfactual reasoning to capture the full picture.