TRIAGE: New framework debugs automatic knowledge graphs for Graph-RAG systems

TRIAGE, created by researchers Axel TahmasebiMoradi, Lucas Schott, and Martin Royer, is a stage-aware instrumentation framework designed to evaluate trustworthiness in automated, document-grounded Graph-RAG systems. As language models increasingly build knowledge graphs automatically via extraction, the risk of cascading failures rises—an error in extraction or graph construction may only surface as a wrong final answer. TRIAGE addresses this by attaching independent, interpretable metrics to three distinct stages: KG Implementation (measuring triple confidence, source coverage, schema, and canonicalization), KG Validation by expert (offline graph-level structural quality, correctness, and completeness using gold annotations), and KG Usage (retrieval coverage, faithfulness, and cost). At deployment, only metrics that need no gold annotations are active; gold-dependent ones serve as offline calibration. The framework creates a diagnostic chain of necessary conditions—if one stage fails, the chain breaks at that point, localizing the failure and recommending which lever to adjust: extraction, graph/schema, or retrieval.

TRIAGE is presented with a proof-of-concept and a reproducible evaluation protocol, making it a practical tool for developers debugging complex Graph-RAG pipelines. By shifting from end-to-end accuracy as the sole metric to stage-specific diagnostics, teams can systematically improve each component. This could significantly boost the reliability of AI applications that depend on dynamically constructed knowledge graphs, such as question answering over large document corpora, enterprise search, and research assistants. The framework's emphasis on trustworthiness and cost (the retrieval cost metric) also helps balance accuracy with operational efficiency.