New Benchmark Exposes AI's Reasoning Flaws, Tests True Retrieval vs. Memorization

Researchers have released HybridRAG-Bench, a new framework designed to test if AI models genuinely reason with retrieved information or just recall memorized facts. It creates benchmarks from recent scientific papers to avoid data contamination, forcing models to perform multi-hop reasoning across both unstructured text and structured knowledge graphs. Initial tests in AI, governance, and bioinformatics show it effectively distinguishes true retrieval-augmented reasoning from simple parametric recall, addressing a critical evaluation gap.