MISA-T router lifts RL rollout throughput 53% for mixed LLM training

Modern RL post-training for LLMs increasingly runs heterogeneous rollout workloads — RLVR, RLHF, and agentic rollouts — on a shared asynchronous inference service. Each has distinct sequence structures, interaction patterns, and KV-residency times, creating wildly different serving demands. Prefix-aware routing helps with cache reuse and load balancing, but it doesn't control how these sessions compete for KV-cache capacity. The result is that a fixed routing policy either starves one workload or distorts the mixture the trainer intended.

To solve this, the authors introduce MISA-T, a routing-layer admission policy that combines three mechanisms: adaptive session admission, workload-aware KV-capacity allocation, and residency-time-aware KV accounting. Evaluations compare MISA-T against a sweep-tuned cache-aware vLLM Router. In rollout-only ablations on Step3.7 and Qwen3.6-35B-A3B, MISA-T improves rollout throughput by 53.3% and 43.6%, respectively, while keeping high prefix-cache hit rates. In a matched 50-iteration Step3.7 experiment, it increases total rollout throughput by 35.6%, reduces mean iteration time by 22.8%, and maintains a consumed workload mixture close to the trainer target — with comparable task scores. The paper is available on arXiv (2608.11152) and targets distributed systems and ML communities.