MetaRoute-Bench benchmarks AI agent routing, hitting 79.4% task success

Agentic AI systems constantly make meta-decisions: should they answer directly, decompose a task, call a tool, execute code, delegate to a specialist, verify an intermediate result, or recover from failure? These choices dramatically impact success, cost, and latency, but most orchestration frameworks hide them behind aggregate task accuracy. MetaRoute-Bench, introduced by Natan Vidra and colleagues, provides an open, inspectable way to compare routing policies under a shared execution model.

The initial benchmark includes 180 synthetic task profiles spanning data analysis, research, and document processing, plus 8 routing policies and 30 paired random seeds. Testing over 43,200 traces, the researchers found that a task-aware compositional policy achieved 79.4% success, beating a strong workload-specific static policy (76.7%), one-shot task routing (67.4%), and direct answering (52.9%). The improvement is 2.7 percentage points (95% CI ±2.0) but costs 4.7% more and adds 6.4% latency. Ablations showed that restricting route composition to a single operation or removing verification causes the biggest performance losses. The results come from a seeded offline execution model, so the authors frame the contribution as a reproducible evaluation method and tradeoff analysis, not proof of production performance. They released task generators, policies, traces, tests, and analysis code to enable live-system validation.