Aligning Dense Retrievers with LLM Utility via DistillationAligning Dense Retrievers with LLM Utility via Distillation

Researchers from the University of Toronto and Layer 6 AI have introduced Utility-Aligned Embeddings (UAE), a new framework that bridges the gap between dense vector retrieval and LLM-based re-ranking for Retrieval-Augmented Generation (RAG). Traditional dense retrievers like BGE-Base excel at speed but suffer from precision limitations, while utility-based methods using LLMs for re-ranking achieve higher accuracy but are computationally prohibitive. UAE solves this by training a bi-encoder to directly imitate a utility distribution derived from perplexity reduction, using a novel Utility-Modulated InfoNCE objective. This injects graded utility signals into the embedding space without requiring any LLM inference at test time, combining the best of both worlds.

On the QASPER benchmark, UAE delivers substantial gains: Recall@1 improves by 30.59%, Mean Average Precision (MAP) by 30.16%, and Token F1 by 17.3% compared to the strong BGE-Base baseline. More impressively, UAE is over 180x faster than efficient LLM re-ranking methods like monoT5, while preserving competitive performance. This speedup makes it feasible to deploy high-quality retrieval at scale, even with limited compute resources. The framework is particularly valuable for enterprise RAG pipelines, where latency and cost are critical constraints. By aligning retrieval directly with generative utility, UAE ensures that retrieved contexts are not just semantically similar but actually useful for downstream generation tasks.