Visual RAG Toolkit: Scaling Multi-Vector Visual Retrieval with Training-Free Pooling and Multi-Stage Search

Researchers have released the Visual RAG Toolkit, a system that drastically improves the efficiency of AI-powered visual search. It tackles the scaling problem of multi-vector retrievers (like ColPali) by using a novel, training-free pooling technique. This reduces the stored vectors per document from thousands to just dozens—a 95% reduction—while maintaining high accuracy. The result is a 4x increase in query throughput, making state-of-the-art visual retrieval far more practical and accessible.