New AI method boosts recommendation accuracy 40% with sparse data
Exponential Reward Weighting (Exp-RSFT) solves noisy feedback in recommender systems
Exp-RSFT, a new fine-tuning method for generative recommender systems, weights each logged interaction by exp(r/λ), using the temperature λ to guard against noisy feedback. The paper shows the method's suboptimality decomposes into coverage and noise costs, with λ balancing the two. On three public benchmarks and a large-scale industrial dataset, performance follows an inverted-U trend as λ changes, while PPO and DPO often over-optimize unreliable reward models and hurt quality. Exp-RSFT improves ranking without online exploration or preference data.
- Exp-RSFT introduces exponential reward weighting (exp(r/λ)) to fine-tune recommendation systems without requiring online exploration or new preference data.
- Achieves up to 40% accuracy improvement over PPO/DPO baselines on 3 public benchmarks and a large-scale Meta dataset.
- Theoretical decomposition shows suboptimality comes from coverage and noise costs, with λ optimizing their tradeoff.
Why It Matters
This method could significantly improve AI-driven recommendation systems in production, reducing noise sensitivity and improving user experience without additional data collection.