SchemaRAG dynamically prunes large schemas for 8.8% better F1 and 47% lower latency

Extracting structured data from unstructured text using LLMs becomes challenging with large, complex schemas. Including the full schema in prompts increases cost, latency, risks the lost-in-the-middle degradation where models forget middle content, and can exceed context length limits. A team of researchers propose SchemaRAG, a retrieval-augmented generation (RAG) framework that dynamically prunes the output schema space. It leverages schema metadata and few-shot examples to reduce the schema before querying the LLM. This approach focuses the model on only the relevant subset of fields, drastically reducing token usage and inference time.

The team evaluated SchemaRAG on real-world healthcare and e-commerce datasets. Results show an 8.8% increase in micro-F1 score (a measure of accurate extraction), a 47% reduction in latency (time to get results), and a 48% reduction in token costs. This makes large-schema extraction practical for production systems. For professionals working with complex form-like data from documents, SchemaRAG offers a drop-in compatible method to improve accuracy while cutting costs and speed. The dynamic pruning eliminates the need to manually curate schema subsets, automating optimization. This approach could be integrated into existing LLM pipelines for tasks like medical record extraction, product cataloging, or any domain with large schemas.