AI Search Gets Faster With a 'Team of Specialists' Trick

When you type a question into an AI assistant or a smart search bar, something has to dig through millions of documents to find the useful ones. That job is called "retrieval." Today it is usually done by a large AI model that reads every question and every document with its full brain switched on — powerful, but slow and expensive. A team of researchers from NYU and elsewhere asked a simple question: what if we used a different kind of AI for this job?

That different kind is called Mixture-of-Experts. Instead of one giant brain handling everything, it works like a company with many specialist departments. For any given question, only the two or three most relevant specialists get pulled in, while everyone else stays idle. The result: far less work per question. In head-to-head tests, these "team of specialists" models beat regular models of similar size by up to 3 points on a widely used search-quality scale, and one of them matched a model nearly twice its active size using 59% fewer working parts — while answering 18% faster.

The team also found a clever money-saving knob. You can quietly reduce how many specialists get consulted at search time — without retraining the model or rebuilding the search index — and still keep more than 99% of the answer quality, with query times dropping by up to 26%. That matters because search speed is what you feel as a user waiting for results, and computing cost is what companies pass along in subscription prices.

One honest caveat: this is a research paper, not a product launch. It has not been turned into a commercial search engine yet, and the gains were measured on academic test sets rather than the messy real-world web. Still, the direction is clear. Smarter, cheaper retrieval means AI assistants that answer quicker, cost less to run, and can be offered to more people — including on smaller budgets and smaller devices.