GRACE-RAG boosts RAG quality 20% with lightweight graph retrieval on mid-scale models
Graph-governed retrieval slashes model size needs for accurate institutional Q&A.
GRACE-RAG (Governed Retrieval Architecture for Canonical Evidence Synthesis) is a new framework from Desai, Kumar, and Devadiga that tackles a key weakness in standard RAG systems: fragmented evidence retrieval in entity-dense, closed-domain institutional settings. Instead of relying on vector-only search and large inference-time reasoning, GRACE-RAG adds a graph-augmented retrieval layer that pre-resolves structural relationships between documents. This externalizes structural reasoning from the generative model to a structured retrieval layer, allowing much smaller, self-hosted models to produce high-quality answers.
In experiments across three model sizes—Mistral 24B, GPT OSS 120B, and Gemini 2.5 Flash—GRACE-RAG achieved consistent improvements in completeness, depth, and anticipatory coverage. The biggest gains (up to 20% overall quality) came from mid-scale models like Mistral 24B, demonstrating that retrieval architecture, not model scale, is the primary driver of structural quality. This means institutions can deploy accurate RAG systems on lightweight, proprietary-free infrastructure with lower compute and latency costs.
- Graph-augmented retrieval layer resolves structural ambiguity offline, eliminating dependence on large, proprietary models.
- Consistent quality gains of up to 20% on mid-scale models (Mistral 24B) across completeness, depth, and coverage metrics.
- Enables self-hosted, closed-domain deployment with reduced computational and latency footprint vs. vector-only RAG.
Why It Matters
Enables institutions to deploy accurate, low-cost RAG systems without relying on large, expensive proprietary models.