Cold-Starts in Generative Recommendation: A Reproducibility Study

A team of researchers from the University of Amsterdam and Shandong University has published a critical study on arXiv (ID: 2603.29845) that systematically examines the 'cold-start' problem in generative AI recommendation systems. The paper, 'Cold-Starts in Generative Recommendation: A Reproducibility Study,' challenges the assumption that modern models built on large pre-trained language models (PLMs) inherently solve the long-standing issue of recommending for new users or newly introduced items. The authors created a unified suite of cold-start protocols to test various generative recommenders, finding that performance claims are often muddled because researchers change multiple key variables—like model scale, identifier design, and training strategy—simultaneously.

The study reveals that while these AI systems can leverage item semantic information (titles, descriptions) and condition on limited user context at test time, cold-start performance is rarely treated as a primary evaluation metric. This lack of focused benchmarking makes it difficult to discern whether reported improvements are due to better architecture or simply larger models. The work serves as a crucial reality check for the field, emphasizing that generative recommendation is not a solved problem and that standardized, transparent evaluation is needed before these systems can be reliably deployed in dynamic, real-world platforms like e-commerce and streaming services.