CIF Feature Selection Ranks Top-4 in Classification, Top-3 in Regression

A new paper on arXiv (2607.01417) by Milletich, Downes, Goley, and Hirst explores Conditional Inference Trees (CIT) and Forests (CIF) as feature-ranking methods. These algorithms reduce split-selection bias by testing features before choosing split thresholds—a computationally heavy process involving repeated permutation tests. The authors rigorously benchmark CIF against 16 other classification methods on 22 datasets and 17 other regression methods on 8 datasets. CIF places 4th in classification and 3rd in regression, showing strong and consistent performance. A key innovation is the use of Bonferroni-corrected +1 Monte Carlo permutation p-values to control nodewise false positive rates under the complete permutation null hypothesis, provided features and permutation budget don't depend on node responses.

The runtime ablation study reveals that adaptive stopping and the number of thresholds searched have the largest impact on fitting time. Turning off adaptive stopping increases time by 4.0–8.4×; using exact threshold search adds 1.9–10.8×. Importantly, downstream prediction scores change by at most 0.011, meaning the speedups come with negligible accuracy loss. However, in sparse high-dimensional settings, the paper notes that forest feature sampling can leave informative features out of many split decisions, potentially limiting recoverability. Overall, the results support CIF as a top-k feature-ranking method for downstream prediction benchmarks, making it a strong choice for practitioners needing reliable feature selection with controlled error rates.