33 AI Search Tools Tested: One Nearly Wins While Running Far Faster

When you ask an AI assistant about your company's documents, it usually doesn't "know" the answer — it looks it up first. That look-up step is called retrieval, and it quietly decides whether the AI gives you something useful or something invented. The better the lookup, the fewer the errors.

A new study from researchers Valentin Velev and Andreas Spitz put 33 of these retrieval systems through a fair race: seven different datasets, equal computing budgets, and no custom tuning for anyone. Think of it as testing 33 cars on the same track with the same amount of fuel. The systems fell into three groups — ones that match keywords, ones that match meaning, and ones that expand your question with related terms.

The results were surprisingly practical. NV-Embed-v2, a heavyweight system, posted the best scores on four of the seven tests, but it took noticeably longer to answer each question. SPLADE-v3, a much lighter keyword-based tool, came close to that top performance while responding far faster — and actually won on MS MARCO, a widely used search test. A third tool, GritLM, led on datasets where the AI must follow specific instructions. The researchers also dug into where each system failed, and found blind spots that suggest everyone could still get better.

The takeaway for anyone paying for AI: bigger isn't automatically better. If your assistant needs to search thousands of documents, a faster system can mean cheaper bills and answers that appear in a blink instead of a pause. And since retrieval quality drives accuracy, picking the right one is one of the simplest ways to reduce confidently wrong answers.