Multi-GPU Hybrid Particle-in-Cell Monte Carlo Simulations for Exascale Computing Systems
New hybrid MPI+OpenMP code enables portable, large-scale plasma physics simulations on Nvidia and AMD GPUs.
A large international research team has published a breakthrough in high-performance computing for plasma physics. Their work, titled "Multi-GPU Hybrid Particle-in-Cell Monte Carlo Simulations for Exascale Computing Systems," presents a new implementation of the BIT1 code that solves critical bottlenecks for running complex simulations on modern, heterogeneous supercomputers. The key innovation is a portable framework that efficiently utilizes thousands of GPUs from different vendors (Nvidia and AMD) by employing OpenMP target tasks with explicit dependencies. This design allows computation and communication to overlap, dramatically reducing idle time and data movement overhead that previously limited scalability.
The team achieved portability and performance through several technical advancements: using persistent device-resident memory, optimizing data into a contiguous one-dimensional layout, and transitioning from unified to pinned host memory to accelerate large data transfers. They also integrated GPU Direct Memory Access (DMA) for direct device-pointer access. For practical use, the framework includes standardized I/O using the openPMD standard and the ADIOS2 library, enabling high-performance file operations, in-memory data streaming, and crucially, in-situ analysis and visualization so scientists can monitor results as the simulation runs.
Performance results demonstrate the framework's capability on pre-exascale and exascale systems. Notably, tests scaled successfully on the Frontier supercomputer (OLCF-5) using up to 16,000 GPUs. This represents a significant leap in run time, scalability, and resource utilization for large-scale PIC-MC simulations, which are fundamental for studying fusion energy, space physics, and advanced material processing. The work, accepted for the International Conference on Computational Science (ICCS) 2026, provides a essential software foundation for the exascale computing era in computational physics.
- Portable multi-GPU framework works on both Nvidia and AMD accelerators using OpenMP target tasks.
- Demonstrated scaling on the Frontier exascale supercomputer using up to 16,000 GPUs.
- Integrated standardized I/O (openPMD/ADIOS2) enables in-situ analysis and high-performance data handling.
Why It Matters
Enables previously impossible large-scale plasma simulations critical for advancing fusion energy research and space physics.