UBEP cuts MoE all-to-all latency by 52% on production superpods

Deploying Mixture-of-Experts (MoE) models on production superpods such as NVIDIA’s NVL72/576 and Huawei’s CloudMatrix384 has been hampered by three fundamental bottlenecks: coarse-grained Bulk Synchronous Parallel (BSP) orchestration that serializes interdependent communication phases, prohibitive synchronization overhead that doesn't scale with high-bandwidth fabrics, and severe load imbalance from distance-agnostic scheduling of irregular token traffic. A team of 19 researchers led by Yipeng Liu introduces UBEP (Unified-Bus Expert Parallelism), a production-ready communication library that rethinks MoE's All-to-All primitives specifically for these unified address space architectures. UBEP eliminates the strict execution serialization, reduces synchronization costs, and schedules token traffic with awareness of network distance, unlocking the full potential of high-bandwidth interconnects.

In large-scale experiments, UBEP achieved up to 52.4% reduction in All-to-All latency and up to 11.1% improvement in MoE inference Time Per Output Token (TPOT). These results demonstrate that raw interconnect bandwidth alone is insufficient; intelligent re-architecture of communication libraries is critical for scaling sparse expert models. UBEP is now available as an open-source library and is designed for easy integration into existing training and inference pipelines, making it immediately valuable for organizations running large MoE models on cutting-edge superpod hardware.