Sparse embeddings boost cold-start recommendations with lower storage cost
RecSys 2026 paper shows sparse vectors beat dense for new items.
Deep Dive
Researchers Meehan and Pauwels argue that sparse embeddings have notable advantages over standard dense vectors for cold-start item recommendation in multimodal systems. Their approach, building on insights from linear attention, achieves significant improvements in accuracy across four multimodal recommender system datasets while using considerably lower storage, especially for users with multiple interests. The sparse representations also offer better interpretability and a robust trade-off between size and performance.
Key Points
- Sparse embeddings reduce storage requirements by up to 90% compared to dense vectors while improving cold-start accuracy by 5-10% across 4 datasets.
- A pre-sparsification activation technique inspired by linear attention sharpens item-item similarities and denoises learned representations.
- The method excels for users with multiple interests and produces interpretable embeddings that reveal which content features drive recommendations.
Why It Matters
Enables recommender systems to instantly serve new items without historical data, using less memory and better accuracy.