Silas Koemen's DiffGBM beats tabular regression benchmarks

Silas Koemen has proposed DiffGBM, a novel approach to probabilistic tabular regression that leverages tree-based diffusion models without relying on neural density estimators. The key innovation lies in making the 'conditioning defaults'—such as noising paths, parameterization, and training distributions—explicit and tunable through a concept called 'score-flex.' This allows practitioners to optimize the model's performance across multiple axes, including residualization, preconditioning, and loss weighting, all within a LightGBM framework.

In empirical evaluations across 11 tabular benchmarks, DiffGBM outperformed the baseline on every dataset under a strict 40-trial tuning budget, achieving an aggregate Continuous Ranked Probability Score (CRPS) of 0.725 compared to the baseline's 0.699. Notably, the flow-matching approach delivered a 5.2x speedup in sampling while maintaining strong calibration, while the stochastic sampler provided higher accuracy at the cost of speed. The method's flexibility and performance suggest it could become a new standard for probabilistic tabular regression in machine learning workflows.