Stellar Slashes Multimodal Retrieval Memory by 100x — And It Could Unlock Scalable RAG
Disk-backed retrieval makes large-scale RAG practical for real-world deployment
Deep Dive
Stellar is a scalable multimodal document retrieval framework that stores token-level document embeddings on disk and loads only candidate embeddings into memory for late interaction. It uses Lexical Representation-based Filtering (LRF) with a fine-tuned Multimodal Large Language Model and Efficient Disk-backed Late Interaction (DLI). Experiments on four benchmarks and a large-scale dataset show it reduces memory overhead and query latency by 1-2 orders of magnitude without compromising retrieval effectiveness.
Key Points
- Stellar stores token-level document embeddings on disk, loading only a small candidate set into memory for late interaction, cutting memory by 10-100x.
- Uses Lexical Representation-based Filtering (LRF) with a fine-tuned MLLM as a sparse encoder to efficiently reduce candidate documents before retrieval.
- Disk-backed Late Interaction (DLI) leverages balanced clustering and a cost model to dynamically load only needed embeddings, slashing query latency by 10-100x.
Why It Matters
Makes large-scale multimodal RAG economically viable by removing the memory barrier for real-world deployments.