Relevance-Based Embeddings Beat Heavy Rankers for Fast Retrieval
Researchers prove embedding queries via expensive ranker scores can approximate any similarity model.
Deep Dive
Researchers Shevkunov et al. introduce Relevance-Based Embeddings, a method that uses scores from an expensive ranking model to improve query and item representations for candidate retrieval. They theoretically prove these embeddings can approximate any complex similarity model under mild conditions. Experiments on diverse academic and production datasets illustrate the power of their method.
Key Points
- Replaces separate embedding training by using heavy-ranker relevance scores as embeddings, enabling fast cosine similarity search.
- Theoretically shown to approximate any complex similarity model under mild conditions, with rigorous analysis of support item selection.
- Outperforms standard two-stage retrieval on academic and production datasets, matching heavy-ranker accuracy while being significantly faster.
Why It Matters
A simpler, more accurate retrieval method that eliminates the need for separate embedding models, cutting costs and complexity.