Bayesian GP framework slashes high-dimensional inputs with built-in reduction

Gaussian Process (GP) modeling struggles with high-dimensional inputs due to the curse of dimensionality. Traditional approaches tackle this by first reducing input dimensions (e.g., via PCA) and then fitting a GP, but this two-stage separation can lose information and degrade uncertainty estimates. A new paper from Eric Herrison Gyamfi and colleagues proposes a unified Bayesian framework that performs dimension reduction and GP inference simultaneously. The method uses a hierarchical Bayesian model with priors defined on the Stiefel manifold, ensuring the projection matrix remains orthonormal during inference. To sample from the complex posterior, the authors employ Hamiltonian Monte Carlo with geodesic flow, which respects the manifold geometry.

The framework is further extended to incorporate Deep Gaussian Processes (DGP), enabling it to handle nonlinear, hierarchical structures in data. Through extensive numerical studies, the authors demonstrate that while the unified Bayesian approach incurs higher computational costs compared to standard two-stage methods, it significantly improves predictive accuracy and provides more reliable uncertainty quantification. This makes it a principled and robust alternative for applications in computational science, engineering, and other fields where faithful uncertainty estimates are critical. The paper is available on arXiv with code links anticipated.