arXiv Study: Meaningful Routing Requires Diverse, Stable AI Societies

In a new preprint on arXiv, researchers Fantine Huot, Michael Kaisers, and Mirella Lapata tackle a fundamental question in multi-model AI systems: when is routing actually meaningful? Current evaluations focus almost exclusively on task accuracy and inference cost, but the authors argue that two orthogonal properties—behavioral differentiation among agents and stability of the routing policy—are essential. Without differentiation, routing over a homogenous model pool is vacuous; without stability, surface-form variants of the same query get assigned to different agents, preventing specialization. The paper introduces a perturbation-based robustness metric and adapts Hierarchic Social Entropy (HSE) to measure diversity in language model societies.

Testing on benchmarks EmbedLLM and RouterBench, the authors uncover striking patterns. HSE exhibits strong diminishing returns: a curated subset of fewer than ten agents recovers most of the diversity present in a large pool, offering a practical coreset heuristic for society design. More critically, they find that accuracy and meaningfulness can sharply diverge. KNN-based routers gain accuracy when societies are specialized, but their robustness collapses under perturbation—meaning small input changes send queries to entirely different models. In contrast, prompted routing remains stable across all perturbation types, even though it may not maximize raw accuracy. The work provides a new lens for building robust, efficient multi-model systems, where routers must balance performance with these deeper properties.