From Skew to Symmetry: Node-Interconnect Multi-Path Balancing with Execution-time Planning for Modern GPU Clusters

Researchers from Ohio State University have introduced NIMBLE (Node-Interconnect Multi-path Balancing with Execution-time orchestration), a novel runtime system designed to solve a critical bottleneck in modern AI training: inefficient communication across GPU clusters. Despite high-bandwidth hardware like NVLink and NDR400 InfiniBand, many AI workloads suffer from traffic skew, where a few links get congested while others sit idle. Traditional frameworks like NCCL and MPI with UCX use static routing that can't adapt to this imbalance, leaving significant performance on the table.

NIMBLE addresses this by dynamically redistributing traffic in real-time to balance utilization across all available intra-node and inter-node paths. It formulates the problem as a capacity-normalized minimum-congestion optimization and solves it efficiently with a multiplicative-weights algorithm. The system is endpoint-driven and integrates transparently with existing communication libraries, requiring no application changes while preserving ordering and determinism.

In practical tests on H100-SXM4 GPU clusters, the performance gains are substantial. NIMBLE achieved up to 2.3x higher intra-node bandwidth and 3.8x higher inter-node throughput compared to single-path baselines. More importantly for AI workloads, it outperformed industry-standard NCCL and MPI by up to 5.2x on skewed All-to-Allv communication patterns and delivered 1.35x speedups on end-to-end LLM Mixture-of-Experts (MoE) training workloads. The system matches baseline performance under balanced traffic, ensuring no regression.

The technology employs CUDA-aware GPU kernel-based RDMA pipelining to route traffic through intermediate GPUs and rail-matched NICs, creating a more flexible network fabric. This approach is particularly valuable for large-scale AI training where communication patterns are often unpredictable and imbalanced, making NIMBLE a potential game-changer for reducing training times and improving cluster efficiency.