New AI Trick Makes Chatbots Answer 45% Faster Without Getting Dumber

When you ask an AI chatbot a question, it usually doesn't just rely on what it memorized during training. It goes and looks things up first — a trick called RAG (letting AI look things up in a knowledge base before answering). That lookup step is what stops it from making things up. The problem: most systems search the same way every single time, whether you asked "what's our return policy?" or "how do these three contracts conflict?"

The first question needs one quick fact. The second needs the AI to connect dots across many documents. Treating them the same wastes time and money on easy questions — and leaves the AI short on context for hard ones. This paper calls that the "static retrieval fallacy." The fix: a small, lightweight classifier reads your question first and decides how deep to dig. It picks between grabbing one fact, spreading outward to nearby related items, or zooming out to the big picture.

Across five different subject areas, this approach kept answer quality steady while cutting the amount of text sent to the AI by up to 48.7% and cutting total wait time by 45.3%. That matters because AI companies pay per chunk of text processed. Less text means cheaper answers, faster responses, and less energy burned in data centers.

This is a research preprint, not a product you can buy today. It also only tested five specialized document collections, so results may differ on messy real-world data. But the direction is clear: the next wave of AI improvement may come from being smarter about where to look, not just from bigger models.