AI 'Traffic Cops' That Pick the Best Chatbot Don't Beat a Simple Fix

When you ask an AI a question, something has to decide which AI model answers it. Cheap, fast models are fine for simple requests; expensive, powerful ones handle the hard stuff. Companies sell 'routers' — software traffic cops — that make that choice automatically, promising you pay premium prices only when truly needed. That promise is worth a lot, because AI bills are mostly driven by which model you use.

A new paper by Kiran N. Kumar and Santhosh K. Saminathan put four free, open-source routers to a fair test. They used the same tasks, the same menu of AI models, and the same rules for all four, running 290 frozen tasks against a locked set of 2,610 possible model answers. Earlier comparisons were muddled because each router got a different setup, making results impossible to line up.

The result was deflating. Three of the four routers handed out nearly identical instructions no matter what the question was — effectively ignoring the prompt. The fourth, vLLM Semantic Router, did respond to the content of each request, but it failed to win any of the four test suites. A dead-simple baseline that always picks the middle-tier model matched a rival router almost exactly. In other words, the gains tracked which tier of models got picked more than any clever matching of task to model.

The practical takeaway: if you are buying AI routing to save money, ask for proof that the savings come from task-specific wins, and compare it against just picking one sensible model. The authors are careful to say their findings cover these specific configurations and tasks, not routing in general — routing may still help at larger scale or with different models. The point is that big claims need strong controls.