DML study: learner choice skews confidence intervals, links rurality to obesity
Coverage drops as sample sizes grow across 5 ML learners in Double ML study.
Xu, Ma, and Xiang's arXiv paper compares analytical and bootstrap confidence intervals for Double Machine Learning (DML) across five learners—OLS, LASSO, Random Forest, LightGBM, and Neural Networks. Simulations show substantial variability in coverage probability depending on learner choice, and in many settings, coverage declines as sample size increases. Applied to U.S. county data on rural-urban differences, the analysis finds greater rurality has a statistically significant increasing effect on obesity prevalence, with model performance still varying by learner choice.
- Coverage probability of DML confidence intervals varies significantly across 5 ML learners (OLS, LASSO, Random Forest, LightGBM, Neural Networks)
- Counter-intuitively, coverage decreases as sample size grows in many settings, for both analytical and bootstrap intervals
- Real-data analysis of U.S. counties shows rurality significantly increases obesity prevalence, but the effect magnitude depends on the choice of learner
Why It Matters
Causal inference reliability hinges on learner selection, not just DML theory—practitioners must check coverage, not just point estimates.