New AI Search Trick Makes Chatbots Answer Faster and Smarter
Your AI assistant wastes effort on easy finds — this fix makes it smarter.
When you ask an AI chatbot a broad question, retrieval systems may need to assemble a large set of documents into a finite LLM context window. Reranking those candidates relies on fast dual-encoders plus more expensive cross-encoders, but the cross-encoder budget is strictly bounded by latency and is often smaller than the context window capacity. That mismatch makes standard reranking structurally flawed: it wastes compute verifying obvious top candidates while ignoring relevant documents further down the initial ranking. A new method called BoundaryMORPH instead allocates cross-encoder budget specifically for the context capacity, using a Gaussian Process that treats the initial dual-encoder ranking as a structural prior and spends cross-encoder calls on resolving top-k set membership at the boundary rather than seeking a single most-relevant document, with information from each call propagating to unscored documents. The authors report that BoundaryMORPH achieves state-of-the-art set retrieval quality across multiple models and datasets with open-ended queries, with +5.4 nCG@100 over the strongest baseline.
- AI chatbots answer questions by looking up documents, but can only read a handful at a time.
- BoundaryMORPH aims expensive double-checking at documents near the cut-off line, not the obvious winners.
- Tests showed a 5.4-point accuracy gain with no extra cost or waiting time.
Why It Matters
Better answers to broad questions, at no extra cost or delay — once chatbot makers adopt it.