Scaled Block Vecchia Approximation for High-Dimensional Gaussian Process Emulation on GPUs
New distributed algorithm handles massive 2.56B-point datasets, enabling large-scale scientific simulation emulation.
A collaborative research team has published a breakthrough in scaling Gaussian Process (GP) models, a cornerstone of statistical emulation for scientific simulations. Their new Scaled Block Vecchia (SBV) algorithm is the first distributed implementation of any Vecchia-based GP approximation, designed specifically for modern GPU clusters. By integrating the Scaled Vecchia method for handling complex data with the Block Vecchia approach to reduce computational complexity, SBV tackles the traditional bottleneck of GP models: poor scalability with large datasets. The implementation uses MPI for communication between nodes and the MAGMA library to accelerate batched linear algebra operations on GPUs.
In practical tests, the SBV algorithm demonstrated impressive performance and scalability. The team successfully applied it to a real-world workload emulating a respiratory disease model with 50 million data points. The system achieved near-linear scaling on massive hardware configurations, running efficiently on up to 512 NVIDIA A100 and GH200 GPUs while processing a staggering 2.56 billion points. Beyond raw speed, the approach also proved to be more energy-efficient than traditional exact GP solvers. This combination of scale, speed, and efficiency establishes SBV as a powerful new framework for tasks like uncertainty quantification and optimization in fields that rely on massive, compute-heavy simulations.
- First distributed Vecchia-based GP algorithm, using MPI and MAGMA for GPU acceleration.
- Scaled to 2.56 billion data points across 512 NVIDIA A100/GH200 GPUs with near-linear efficiency.
- Tested on a 50M-point respiratory disease model, reducing energy use vs. exact solvers.
Why It Matters
Enables uncertainty quantification and optimization for massive scientific simulations (e.g., climate, disease models) that were previously computationally infeasible.