StrataCL boosts AI supernode comms by 1.6x on CloudMatrix384
Zero-redundancy fabric-native library slashes data copies and unlocks NPU-driven offloading.
Modern distributed AI workloads running across hundreds of accelerators face a major bottleneck: communication. Existing libraries manage user and communication buffers separately, leading to redundant data copies or costly user-buffer registration. StrataCL, developed by a team from Huawei and academic collaborators, tackles this with a zero-redundancy, fabric-native approach. Its key innovation is ‘registration-on-allocation,’ which directly enables user-buffer communication without extra copies. It also introduces workload-balanced NPU-core partitioning and NPU-driven SDMA (shared memory direct access) offloading to fully exploit the supernode’s architecture. The result is a communication layer that eliminates overhead while maximizing hardware utilization.
Tested on Huawei’s CloudMatrix384 supernode, StrataCL delivers impressive gains: collective bus bandwidth improves by up to 1.6×, and MoE dispatch/combine bandwidth by 1.4×. Across three production workloads — LLM inference, LLM training, and Recsys training — the library shows 1.9× higher inference throughput, 2.2× lower P99 time-to-first-token (TTFT), and 1.4× / 1.3× reductions in training iteration time. These numbers demonstrate that by removing communication bottlenecks, StrataCL can significantly accelerate both training and serving for the largest AI models.
- Registration-on-allocation eliminates redundant data copies between user and communication buffers.
- Workload-balanced NPU-core partitioning and NPU-driven SDMA offloading fully exploit supernode architecture.
- On CloudMatrix384: 1.6× collective bus bandwidth, 1.9× LLM inference throughput, 2.2× P99 TTFT reduction.
Why It Matters
StrataCL removes communication bottlenecks, enabling faster training and inference for large-scale AI models on supernodes.