Alibaba's Libra speeds up long-context LLM training by 2.54x

Long-context LLM training faces a fundamental load-balancing problem: packing sequences into fixed token lengths balances memory but creates severe attention workload skew. Because attention cost scales quadratically with sequence length, equally-sized packed sequences from long-tailed corpora can have vastly different compute demands, causing stragglers in data-parallel training and pipeline bubbles. Existing solutions either balance at coarse granularities (sequences or microbatches) where outliers dominate, or disaggregate attention across a global worker pool whose communication domain grows with the data-parallel degree.

Libra tackles this by operationalizing the law of large numbers: it groups packed sequences and their context-parallel groups into fixed-size "sequence pools" that remain bounded as the system scales out. A Variance-Reduced Sequence Placement algorithm co-locates sequences with complementary attention workloads to minimize inter-pool skew. Inside each pool, Tiled Attention Pooling dispatches fine-grained sequence-head tiles across GPUs while a pipelined runtime overlaps tile exchange with attention computation. On Qwen3-Turbo training at 256K and 1M token contexts, Libra achieves up to 2.54x end-to-end throughput improvement over Ulysses, with a 3.14x worst-step straggler-attention speedup in microbenchmarks. The system has been deployed in production for hundreds of thousands of GPU-hours on jobs ranging from 32K to 1M tokens, preserving training semantics with zero changes to model layers, optimizers, or pipeline schedules.