LP-FFT-RFT framework boosts recommender foundation models in live A/B tests

Recommendation foundation models excel at modeling long-horizon user behavior, but adapting them to real-world serving surfaces usually relies on Supervised Fine-Tuning (SFT) with task-specific objectives like clicks or likes. That doesn't necessarily align with the business metrics that determine recommendation quality. To fix this, researchers from a team including Oseong Choi, Hoeinn Kim, Jihoon Lee, Byungsoo Kang, and Taeyeong Jang introduce a three-phase post-training framework that explicitly separates downstream adaptation from business-metric alignment.

The first phase, Linear Probing (LP), stabilizes randomly initialized downstream heads within a frozen pretrained representation space. Phase two, Full Fine-Tuning (FFT), jointly specializes the entire model for the target task. On top of this stabilized policy, the third phase applies Reinforcement Fine-Tuning (RFT) to align with practical business objectives using a learned reward model. Crucially, the policy is trained on dense implicit feedback, while sparse business-metric supervision is used only for reward modeling. Offline experiments show the progressive LP-FFT-RFT pipeline outperforms single-phase alternatives, and using reward-based alignment yields a stronger serving policy than directly using the reward model for ranking. Large-scale online A/B tests further confirm production quality improvements over a conventional non-foundation baseline. The work is accepted to RecSys '26 Industry Track, with a reference implementation available online.