Yandex study: GNN pretraining helps small models, not large recommendation systems

Yandex researchers (Makeev et al.) conducted a large-scale industrial case study comparing two item embedding strategies for transformer-based sequential recommenders: pretrained graph neural network (GNN) embeddings versus end-to-end trainable ID embeddings. They evaluated across three production systems at Yandex: Yandex Market (e-commerce), Yandex Music (streaming), and a low-resource dataset from Yandex Lavka (grocery delivery). The study is unique in its scale and direct cost-quality comparison in a real-world setting.

The key finding: for the low-resource Lavka dataset, pretrained GNN embeddings significantly outperformed end-to-end learning. However, for the large-scale Market and Music systems (with billions of interactions), the GNN pretraining stage provided no worthwhile quality lift over standard ID embeddings. This suggests that when training data is abundant, the transformer can learn effective item representations during training, making the separate GNN pretraining stage unnecessary and adding cost. The paper includes publicly available code and data for the Lavka experiment to facilitate reproducibility and further research.