On Strengths and Limitations of Single-Vector Embeddings

A team of researchers including Archish S and Mihir Agarwal published a groundbreaking paper analyzing why single-vector embedding models—the backbone of modern RAG (retrieval-augmented generation) systems—fail dramatically on complex retrieval tasks. Their work builds on the LIMIT dataset introduced by Weller et al. in 2025, which exposed substantial performance drops in popular models. The researchers systematically debunked the theory that limited dimensionality alone causes these failures, showing mathematically that 2k+1 dimensions should suffice for top-k retrieval.

Instead, they identified domain shift and misalignment between embedding similarities and task relevance as primary culprits. While finetuning single-vector models on LIMIT-like data improved recall, it caused catastrophic forgetting—performance on standard benchmarks like MSMARCO dropped by more than 40%. In contrast, multi-vector models showed minimal forgetting. The team's mathematical analysis revealed single-vector models are particularly susceptible to the 'drowning in documents' paradox, where relevant information gets lost as corpus size grows.

This research provides the first comprehensive framework explaining why single-vector embeddings struggle with real-world complexity. Through controlled experiments and toy mathematical models, the authors demonstrate how embedding similarities act as noisy statistical proxies for relevance, fundamentally limiting single-vector approaches. Their findings suggest current embedding architectures need rethinking for reliable enterprise applications.