MCompassRAG adds topic metadata for 8% better RAG retrieval at 5x lower latency

Retrieval-augmented generation (RAG) systems face a fundamental trade-off: fine-grained chunks improve precision but balloon the search space, while larger chunks reduce candidates but introduce semantic noise from mixed topics. MCompassRAG, introduced by Amirhossein Abaskohi and collaborators, solves this by enriching chunk embeddings with topic-level metadata in the same embedding space. Instead of relying solely on cosine similarity between queries and noisy chunk vectors, the framework uses a lightweight retriever trained via LLM-teacher distillation to perform topic-aware retrieval without additional LLM calls at inference time. This approach acts as a semantic compass, guiding the system to relevant evidence even across large, heterogeneous corpora.

In evaluations across six complex retrieval benchmarks, MCompassRAG achieved an average 8.24% improvement in information efficiency (IE) while operating with over five times lower latency than the strongest efficient RAG baselines. The method is particularly impactful for deep research tasks that require both speed and precision. By shifting the burden from expensive LLM calls to a small, distilled model, MCompassRAG makes high-quality retrieval practical for production systems. The code is publicly available on GitHub, allowing teams to integrate topic-aware retrieval into their own RAG pipelines without sacrificing performance or incurring prohibitive costs.