Google researchers use LLM personas for real-time video recs at scale

Researchers from Google (including Minmin Chen, Lichan Hong, and Ed H. Chi) have published a paper detailing a novel framework that leverages large language models (LLMs) to generate real-time, natural-language user interest personas for a large-scale commercial video recommendation platform—widely believed to be YouTube. Existing LLM-based approaches often rely on structured IDs or offline processing, limiting semantic richness, real-time adaptability, and interpretability. The new method addresses these gaps by generating dynamic persona descriptions during serving, directly capturing a user's current interests while also suggesting novel topics to balance the classic exploitation-exploration trade-off.

To overcome the enormous computational cost of running LLM inference for a billion users, the team designed a cost-efficient architecture combining knowledge distillation (training a compact model from a larger teacher LLM), asynchronous inference pipelines, and input optimization via semantically clustered video representations. This allows real-time persona updates without prohibitive latency or cost. Extensive offline evaluations, user studies, and live A/B tests demonstrated significant improvements in viewer value metrics. The work represents a concrete step toward more dynamic, explainable, and satisfying personalized experiences at industrial scale.