Spectral Tempering for Embedding Compression in Dense Passage Retrieval

Researchers from the University of Amsterdam have introduced Spectral Tempering (SpecTemp), a breakthrough method for compressing dense passage retrieval embeddings without sacrificing accuracy. Traditional approaches like PCA and whitening face fundamental trade-offs: PCA preserves dominant variance but underutilizes capacity, while whitening enforces uniformity at the cost of amplifying noise. SpecTemp bridges these extremes by analyzing the eigenspectrum of retrieval embeddings and applying adaptive dimension scaling based on local signal-to-noise ratios.

The key innovation is that SpecTemp automatically determines the optimal scaling strength γ(k) for each target dimensionality k, eliminating the need for task-specific hyperparameter tuning. Unlike previous methods that treated γ as a fixed parameter requiring expensive grid searches, SpecTemp derives adaptive scaling directly from corpus statistics using knee-point normalization. This makes it completely learning-free—requiring no labeled data or validation sets—while achieving performance within 1-2% of oracle values obtained through exhaustive search.

Extensive experiments demonstrate that Spectral Tempering works consistently across different models and datasets, making it particularly valuable for deploying retrieval-augmented generation (RAG) systems at scale. By reducing embedding dimensions while maintaining retrieval quality, the method enables more efficient storage and faster similarity searches in production environments. The researchers have made their code publicly available, allowing immediate implementation in existing dense retrieval pipelines.