Analyzing Reverse Address Translation Overheads in Multi-GPU Scale-Up Pods

A team from AMD, Google, and the University of Wisconsin-Madison has published groundbreaking research identifying a previously overlooked performance bottleneck in multi-GPU AI systems. Their paper, 'Analyzing Reverse Address Translation Overheads in Multi-GPU Scale-Up Pods,' reveals how emerging high-speed interconnects like NVLink and UALink introduce a critical translation step when GPUs directly access each other's memory. This 'Reverse Address Translation' converts Network Physical Addresses to System Physical Addresses, creating significant overhead that wasn't well understood until now.

Using an extended ASTRA-sim framework with Omnet++ network modeling, the researchers simulated Link MMUs and TLBs across varying GPU counts and input sizes. They discovered that cold TLB misses dominate latency for small, latency-sensitive collective operations, causing up to 1.4x performance degradation. Larger collectives benefit from warmed caches but still face scalability challenges. The team proposes two innovative solutions: fused pre-translation kernels that overlap translation with computation, and software-guided TLB prefetching to proactively populate likely-needed entries.

This research establishes a foundation for designing more efficient destination-side translation mechanisms in large-scale AI systems. As AI models grow exponentially in size and complexity, optimizing communication between GPUs becomes increasingly critical for both training and inference workloads. The proposed optimizations could significantly improve throughput and scalability for next-generation AI infrastructure, potentially saving millions in compute costs for organizations running large-scale distributed AI workloads.