ColBERTSaR shrinks ColBERT indexes by 70% via product quantization

ColBERT is a popular neural retrieval architecture, but its index requires 5–10 times the disk storage of raw text, limiting scalability. At query time, gathering and decompressing document token embeddings are major inefficiencies. To solve this, researchers (Eugene Yang, Andrew Yates, Dawn Lawrie, James Mayfield, Saron Samuel, Rohan Jha) propose ColBERTSaR, an embedding quantization approach that turns a ColBERT index into a true inverted index. Theoretically, this is equivalent to learned-sparse retrieval except for the scoring mechanism. Empirically, ColBERTSaR produces indexes 50–70% smaller than a one-bit PLAID index—the current standard—while retaining retrieval effectiveness.

The method uses product quantization to compress token embeddings, enabling a sparse inverted structure that eliminates the need for full gathering and decompression of document embeddings at query time. This drastically reduces both storage requirements and query latency. The paper, accepted at SIGIR 2026 as a short paper, demonstrates that the approach does not sacrifice accuracy. For teams deploying neural search at scale, ColBERTSaR offers a practical path to handle larger document collections without exploding infrastructure costs.