MITRE-SAGE: Multi-agent RAG beats LLMs in cybersecurity QA with 3,000-question benchmark

MITRE-SAGE, a new multi-agent RAG framework from Ali Habibzadeh, Farid Feyzi, and Reza Ebrahimi Atani, tackles cybersecurity QA where large language models struggle with domain knowledge gaps, hallucinations, and capturing both semantic and structural relationships. The system decomposes complex tasks into three specialized roles: query interpretation, evidence retrieval, and answer synthesis, enabling more reliable and interpretable responses. By integrating structured security knowledge with semantic retrieval, MITRE-SAGE supports vulnerability assessment, threat profiling, and relationship extraction without requiring massive single-model retraining.

The researchers also introduce MITRE-QA, a benchmark of 3,000 question-answer pairs spanning diverse cybersecurity knowledge tasks. In head-to-head evaluations, MITRE-SAGE consistently outperforms standalone LLMs and conventional RAG baselines. Notably, a lightweight configuration using Qwen2.5-7B sub-agents orchestrated by a Qwen2.5-14B model achieved superior results on five of the eight benchmark tasks, demonstrating that a cost-effective multi-agent architecture can beat larger monolithic approaches. This suggests scalable, interpretable AI for security operations centers fighting alert fatigue and information overload.