SGLang delivers 5x throughput boost for DeepSeek-V4 on GB300

Since the Day-0 launch of DeepSeek-V4 on GB300, SGLang has coordinated a set of kernel, runtime, and hardening improvements that collectively deliver a 5x throughput increase at the same user-visible interactivity. On the SemiAnalysis InferenceX GB300 disaggregated lane (8K/1K, FP4), the June 2026 MTP curve reaches ~11,200 tok/s/GPU at roughly 50 tok/s/user, up from ~2,200 tok/s/GPU on the Day-0 no-MTP curve. The no-MTP and MTP curves both improved across the entire interactivity range, with MTP sustaining 2.6x more throughput at 80 tok/s/user — the range most deployments target. On Blackwell Ultra aggregated lanes, throughput improved by 2.91x at 30 tok/s/user and peak no-MTP throughput jumped over 6x.

Key optimizations include rewiring the DeepSeek-V4 MHC path with fused implementations (DeepGEMM-backed flows, fused RMSNorm, dedicated fused hc_head kernel to reduce intermediate tensor traffic), KV Compression V2, W4A4 MegaMoE alignment, stronger SWA budgeting and eviction, breakable CUDA graph support in the prefill path, and bug fixes in SGLang and Dynamo that removed instability. The Blackwell Ultra lane also moved to a stronger recipe family with better dispatch, sustainable batch sizes, and a mature FP4/MoE path. These changes turn the already-functional Day-0 stack into a production-ready serving solution.