OpenMP Offloading hits 4x multi-GPU speedup across NVIDIA, AMD, Intel

As supercomputers increasingly pack four or more GPUs per node, researchers need efficient ways to exploit multi-GPU systems without rewriting code for each vendor. Ezhilmathi Krishnasamy's new paper, posted on arXiv (2608.11882), tackles this directly by evaluating OpenMP Offloading—a directive-based programming model that works across NVIDIA, AMD, and Intel GPUs—for a 3D heat transfer simulation that includes both computation and halo communication.

The study systematically compares OpenMP implementations using different low-level APIs for memory allocation and transfers (synchronous, asynchronous, and peer-to-peer) against native programming models: CUDA (NVIDIA), HIP (AMD), and SYCL (Intel). Key results show that OpenMP Offloading achieves roughly 2x speedup when using 2 GPUs and around 4x speedup with 4 GPUs compared to single-GPU OpenMP runs, across all three GPU types. This demonstrates that a single portable codebase can scale effectively in multi-GPU nodes, closing the gap with vendor-specific approaches while saving development effort. The paper provides a practical benchmarking framework for scientists and engineers choosing programming models for heterogeneous HPC clusters.