Moebius enables runtime parallelism switches for MoE model serving

Serving massive Mixture-of-Expert (MoE) models efficiently is a challenge because the optimal parallelism strategy depends on request concurrency. Tensor parallelism (TP) excels at low concurrency, while expert parallelism (EP) performs better under high load. Production workloads continually cross this boundary—bursty online traffic followed by quiet periods, or reinforcement learning rollouts that start as high-concurrency bursts and decay. Pinning one layout sacrifices performance when the workload shifts.

Moebius, introduced by researchers from (institution not explicitly stated in provided text, but authors: Shaoyu Wang, Yizhuo Liang, Jaeyong Song, Chong Li, Seo Jin Park), solves this by switching between EP and TP at runtime without restarting the engine or dropping in-flight requests. The key insight: EP and TP are just two layouts of the same model—they compute identical functions over byte-identical expert weights and KV cache. Switching only requires moving the slices that change ownership. Modern GPU interconnects make this fast enough to happen between decode steps. On 8x H200 GPUs serving Qwen3-235B-A22B, Moebius matches the best static parallelism at every operating point and improves RL rollout performance by 1.16–1.25x across steps. Each switch takes just 215–434 ms, with only 2.4% memory overhead to hold both layouts resident.