GraphGP scales Gaussian processes to billions of parameters on GPU

GraphGP, developed by Benjamin Dodge, Philipp Frank, and Susan E. Clark, tackles the O(N³) computational bottleneck of Gaussian processes using Vecchia's sparse precision matrix approximation. By conditioning each point on only its k nearest neighbors, the method achieves linear O(N) time and memory, scaling up to nearly a billion parameters on GPU. The authors introduce a bit-reversed k-d tree ordering that maximizes batch parallelism while enabling efficient neighbor searches. Their differentiable CUDA implementation is substantially faster and more memory efficient than a pure JAX baseline, providing building blocks for inference such as forward generation, inverse application, log-determinant computation, and kernel parameter derivatives.

The paper, accepted to the Conference on Physics and AI at Stanford University (PAI 2026), demonstrates GraphGP's ability to handle arbitrary point distributions over a large dynamic range. This breakthrough makes Gaussian processes practical for massive datasets in fields like astrophysics, machine learning, and scientific computing. By offering linear scalability and GPU acceleration, GraphGP opens the door to applying Gaussian processes to problems previously intractable due to memory or compute limits, such as real-time spatial modeling or large-scale uncertainty quantification.