Research reveals BART's secret: It's secretly a Gaussian Process
BART's Markov Chain Monte Carlo estimation may become obsolete as researchers prove it converges to a Gaussian Process with infinite trees.
A groundbreaking paper by Cory McCartan and Melody Huang at arXiv (2607.28844) has revealed a profound theoretical insight: Bayesian Additive Regression Trees (BART)—a powerhouse in machine learning for prediction and causal inference—converges to a Gaussian Process (GP) as the number of trees approaches infinity.
The researchers demonstrated that in this limit, BART behaves like a GP with a specific kernel, offering favorable inferential properties that explain its empirical success. To make this theory practical, they introduced *random tree features*, which approximate the limiting GP and enable ridge regression with minimax-optimal learning rates that grow only logarithmically with input dimension. Crucially, this approximation bypasses the computational burden of traditional MCMC estimation, unlocking faster inference and broader integration into models that use linear predictors. The work not only demystifies BART’s performance but also enhances its scalability and flexibility for real-world applications.
- BART converges to a Gaussian Process as the number of trees grows to infinity, explaining its strong empirical performance
- Introduced random tree features enabling ridge regression with learning rates that depend logarithmically on input dimension
- Approximation bypasses MCMC, reducing computational costs and expanding BART’s use beyond traditional Bayesian frameworks
Why It Matters
This work bridges theory and practice, making BART faster, more scalable, and deployable in modern ML pipelines without sacrificing accuracy.