CADFT: New fine-tuning method slashes variance in LLM instruction tuning

Supervised Fine-Tuning (SFT) has long been the go-to method for aligning large language models, but it suffers from optimization instability and limited generalization when faced with diverse instruction data. A previous fix, Dynamic Fine-Tuning (DFT), addressed token-level gradient scaling, but it still assumed all training demonstrations are equally suitable — an assumption breached by the heterogeneity of large-scale datasets. Now, a new paper from researchers Yucheng Zhou, Junwei Sheng, Qianning Wang, and Jianbing Shen, accepted at ACL 2026, introduces Compatibility-Aware Dynamic Fine-Tuning (CADFT) to fix the sample-level variance problem.

CADFT introduces a dynamic, policy-dependent compatibility signal — computed from the model's own likelihoods — that modulates supervised updates. If a demonstration is incompatible with the current policy, its gradient contribution is suppressed, preventing high-variance updates. For persistently incompatible examples, CADFT employs a delayed, low-frequency rewriting strategy that transforms those samples into learnable targets. The paper shows that CADFT can be interpreted as a variance-controlled estimator, generalizing DFT's token-level stabilization to the sample level. Experiments demonstrate improved training stability, better generalization, and notably, stronger initialization for cold-start reinforcement learning — all while remaining fully supervised and independent of any explicit reward modeling.