Permutation-preserving Functions and Neural Vecchia Covariance Kernels

Gaussian processes (GPs) are powerful for probabilistic modeling but often struggle with scalability and limited kernel expressiveness. The new paper by Cao, Liu, and Lin tackles this by introducing a regression-type parameterization derived from Vecchia approximations. Instead of specifying a kernel function manually, they learn the covariance structure directly by modeling kriging coefficients and conditional standard deviations through deep neural networks. These quantities deterministically characterize the covariance, providing stable learning targets. A key innovation is recognizing the permutation-equivariant structure of conditioning sets in the Vecchia factorization. The authors derive a universal representation for permutation-preserving functions and design neural architectures that respect this symmetry, leading to improved training stability, data efficiency, and better generalization.

The resulting Neural Vecchia Covariance Kernels (NVCK) achieve expressive, non-stationary, and heteroscedastic kernel learning without sacrificing the O(n) computational complexity characteristic of Vecchia approximations. This framework effectively bridges classical GP theory with modern deep learning, allowing practitioners to leverage the flexibility of neural networks while retaining the uncertainty quantification benefits of GPs. The approach is particularly impactful for large-scale spatial statistics, time series, and machine learning tasks where traditional GP kernels are insufficient. By learning covariance directly from data in a scalable way, NVCK opens new possibilities for automated kernel discovery and robust uncertainty modeling in high-dimensional settings.