Rethinking ANN-based Retrieval: Multifaceted Learnable Index for Large-scale Recommendation System
A new learnable index improves cold-content delivery by 57% and eliminates costly ANN search at serving time.
Deep Dive
Researchers from multiple institutions propose the MultiFaceted Learnable Index (MFLI), a unified framework for embedding and indexing. It co-trains a hierarchical codebook with item embeddings, enabling real-time updates and eliminating Approximate Nearest Neighbor (ANN) search during serving. On real-world data with billions of users, MFLI improved recall by up to 11.8% and cold-content delivery by 57.3% compared to state-of-the-art methods.
Why It Matters
This could significantly reduce the computational cost and latency of large-scale recommendation systems used by major platforms.