Researchers unveil MarginMerge for AI document retrieval compression

Researchers from the University of British Columbia's Media and Graphics Interdisciplinary Centre (a collaboration between UBC and the University of Victoria) have published a groundbreaking method for compressing multi-vector visual document retrievers. Their paper, titled 'Coverage Matters: MarginMerge for Compressing Multi-Vector Visual Document Retrievers,' introduces MarginMerge as a novel approach to document retrieval compression that prioritizes query-relevant coverage over traditional patch selection methods.

The technique addresses a critical bottleneck in visual document retrieval systems like ColPali and ColQwen2.5, which achieve strong results by storing fine-grained patch embeddings but suffer from massive index sizes and expensive late-interaction scoring. MarginMerge solves this by selecting coverage-aware anchors, clustering document patches, and using a lightweight shared network to synthesize representative embeddings. The method achieves an average 5-10% vector retention while preserving 97-99% of average nDCG@5 accuracy, reducing storage needs by 90-95% compared to uncompressed indexes.