INFRAMIND boosts multi-agent LLM accuracy 7.6pp, cuts latency 7x

Existing multi-agent LLM orchestration methods select models and topologies based solely on task and model features, ignoring the runtime state of shared GPU clusters. This blindness leads to systematic resource underutilization: preferred models accumulate deep request queues while equally capable alternatives remain idle. In multi-agent pipelines, where each query triggers sequential model calls, these delays compound across downstream steps. To solve this, researchers Ahasan Kabir, Jiaqi Xue, Mengxin Zheng, and Qian Lou from the University of Central Florida propose INFRAMIND, a framework that makes the entire multi-agent stack infrastructure-aware.

INFRAMIND comprises three components: an infra-aware planner that conditions topology and role selection on real-time system load and remaining budget, biasing toward simpler graphs under congestion; an infra-aware executor that observes per-model queue depths, cache utilization, and response latencies at each step to decide which model to call and how deeply to reason; and a budget-aware scheduler that reorders each model's queue so urgent requests are served first. Cast as a hierarchical constrained MDP and solved end-to-end via reinforcement learning, the system learns to balance quality against latency automatically. Across five benchmarks, INFRAMIND delivers up to +7.6 pp accuracy over prior baselines at low load with up to 7x lower latency, and sustains 99.9% SLO compliance under high load, where every baseline drops below 50%. The work highlights a critical blind spot in current agent orchestration and offers a practical solution for production deployments on shared GPU infrastructure.