Centroid initialization boosts LLM recommendation accuracy by 16% with zero overhead
This simple trick recovers semantic structure in LLM-based recommendations, improving Recall@5 by up to 60% for cold items.
A new paper proposes a simple, parameter-free intervention for LLM-based generative recommendation: instead of initializing semantic ID (SID) token embeddings as random Gaussian vectors, initialize them from their corresponding semantic centroids. This preserves the SIDs' continuous geometry and improves pure-SFT Recall@5 by up to 16%, reaches peak performance with up to 40% fewer SFT steps, and improves cold-item Recall@5 by up to 60%—using just a few lines of code and no additional training or inference overhead.
- Random SID token initialization causes embeddings to converge on item popularity rather than semantic similarity, degrading generalization.
- Centroid initialization (mapping SIDs to semantic embedding centroids) improves pure-SFT Recall@5 by up to 16% and cold-item Recall@5 by up to 60%.
- The method is parameter-free and drop-in: a few lines of code, no extra training/inference overhead, and 40% fewer SFT steps to peak performance.
Why It Matters
This makes LLM-based recommendation systems more accurate and efficient, especially for new or niche items, without any extra computational cost.