Your AI Test Scores Might Be Hiding the Truth

A new tool called BenchMIRT is digging into how AI models are really being tested — and it’s finding that many benchmarks are grading things they weren’t designed to measure.

Created by Allen Institute for AI, BenchMIRT takes a closer look at individual test questions, not just overall scores. For example, a safety test might include a question that actually requires strong reasoning to answer correctly — not safety awareness. That means a model could score low on ‘safety’ when it’s really struggling with reading comprehension. The tool uses advanced math (from psychology!) to separate different abilities, like sorting safety from general smarts.

The team tested 100 different AI models across 16 benchmarks totaling over 34,000 questions. Surprisingly, some tests labeled as ‘safety’ were actually measuring reasoning ability. The well-known BBQ bias test, for instance, leaned more on reasoning than safety. Meanwhile, a test about dangerous knowledge (like cyberattacks) turned out to be better at measuring general knowledge than safety caution.

This matters because AI benchmarks shape what models are built to do and how we trust them. If a model gets a high safety score but only because it’s really good at reasoning, it might still fail in real-world safety situations. BenchMIRT helps researchers see the real strengths and weaknesses behind the numbers.