Pinterest's PRL-PUTS uses RL to tune recommendations per user, boosting engagement

Large-scale recommender systems often combine multiple objectives (e.g., relevance, engagement, diversity) into a single utility score using manually tuned weights. Pinterest found this manual tuning to be slow, globally applied, and hard to adapt as business priorities shift. To solve this, a team of researchers from Pinterest introduced PRL-PUTS (Production-Ready RL for Personalized Utility Tuning with Pareto Sweeping). The framework casts utility weight selection as a one-step, value-based RL problem: given request context (user, device, time, etc.), an agent selects a utility-weight vector that re-weights ranker predictions to maximize request-level engagement rewards. PRL-PUTS runs in parallel with ranking inference, adding no additional serving latency. It also includes an inference-time Pareto frontier sweeping mechanism via a scalarization parameter, which produces a family of policies and an empirical Pareto frontier. This serves as a governance artifact, allowing decision makers to instantly view trade-offs and update the deployed operating policy without retraining.

PRL-PUTS was validated both offline using unbiased exploration logs and online in A/B tests on Pinterest’s Homefeed. The online experiments showed a statistically significant +0.13% increase in successful sessions, a core engagement metric. This improvement, while modest in percentage, is meaningful at Pinterest’s scale. The framework is ranker-independent, meaning it can be applied to any existing ranking model without changes. By replacing manual weight tuning with a learned, context-aware policy, Pinterest demonstrates a practical path toward more personalized and adaptive recommender systems. The Pareto sweeping feature also provides transparency, enabling product teams to quickly adjust trade-offs (e.g., favoring engagement over diversity) without engineering overhead.