DRIFT algorithm refines training data to push LLM capability ceiling

Researchers from multiple institutions introduce DRIFT (Data Refinement via On-Policy Influence Functions for Supervised Fine-Tuning), a novel approach that optimizes training data distribution to push the capability upper bound of large language models. Standard data curation methods excel at preserving performance under limited budgets but fail to actually improve the final model. The team identifies two root causes: a proximity gap from using off-policy validation targets, and a severe bias toward gradient norm in standard influence functions. DRIFT replaces external reference data with the model's on-policy rollouts as validation targets, closing the proximity gap, and adds signed weighting based on trajectory correctness to counter gradient hacking.

Experiments on 7B-parameter instruction and reasoning models show that DRIFT consistently outperforms existing data curation baselines, raising the performance ceiling on both types of models. The method allows a small set of validation queries to act as reliable anchors for attributing the entire dataset, making it computationally practical. This work represents a shift from simply accelerating training under constraints to actively refining data distribution for maximum capability gains. The code and paper are available on arXiv under ID 2606.18307.