QXGB delivers quantile regression for XGBoost with near-zero crossing

Most machine learning algorithms predict a single point estimate, offering little insight into uncertainty or the conditional distribution of outcomes. To fix this, Mandy Yao and Meredith Franklin from University of Toronto introduced QXGB, a quantile-based gradient boosting framework that integrates a convolution smoothed quantile loss directly into XGBoost. The method derives the gradients and Hessians needed for different kernel specifications, restoring the second-order information XGBoost relies on for tree splitting. This lets users estimate conditional quantiles efficiently, then construct dense cumulative distribution functions (CDFs), exceedance probabilities, and tail behavior for extreme events—without sacrificing XGBoost's speed.

In simulated benchmarks, QXGB outperformed alternative smoothed quantile losses, the native quantile objective in the XGBoost Python package, and independent versus multi-output tree estimation. The team also applied QXGB to predict fine particulate matter (PM2.5) in northern California, specifically capturing elevated levels from wildfire smoke. Results show near-zero quantile crossing, well-calibrated CDFs and exceedance probabilities, and useful tail characterization for extremes. For data scientists, this means a practical, drop-in way to get probabilistic predictions and risk-aware forecasts from a popular gradient boosting library, with interval estimation as a bonus.