FlashBoot loads large models in 0.32 seconds—270x faster at rack scale

As flagship MoE models balloon in both total parameters and expert count, getting them onto GPUs quickly has become a serious bottleneck in elastic serving. FlashBoot, a new weight-loading subsystem from researchers Issac Zhu, Jason Zhao and colleagues, attacks three structural inefficiencies that keep today's loaders far below link bandwidth: fragmented per-tensor memory objects, NCCL communicator setup that costs 10-110 seconds before a single byte moves, and a serial cross-node GPU clone path.

FlashBoot's core is FabricArena, a contiguous, exportable, inter-node-addressable tensor memory layout built on SGLang. On top of it, FlashLoad moves weights from CPU as a single bulk zero-copy transfer, while FlashClone replicates a resident model from a remote GPU via remote-memory mapping that skips NCCL setup entirely. In tests on NVIDIA GB300 NVL72 with DeepSeek-V4-Pro and DeepSeek-V4-Flash, FlashClone mapped remote weights in ~10ms (versus 10-110s for NCCL) and sustained at least 700 GB/s per clone. End-to-end, FlashBoot accelerated single-node weight loading up to 50x (20.1s → 0.4s) and concurrent rack-level loading by more than 270x (87s → 0.32s). Code will be released publicly.