AI Report Cards Are Misleading, Study Finds — Here's What to Trust

AI "agents" — AI that can take actions on its own, like booking, coding, or browsing for you — get compared using standardized tests called benchmarks. Companies use those scores to decide which AI to buy and deploy. But a new study argues that those scores often measure the test setup as much as the AI itself. Think of it like a cooking contest where a helper secretly chops all the vegetables.

The paper identifies two problems. First, "scaffolding": pre-written helper code that handles the tricky steps. When the helper makes a decision, the score partly belongs to the people who wrote the helper, not the AI. Second, scoring: many tests grade whether an answer is shaped correctly — the right format, the right file, the right length — instead of whether it is actually correct. That is like grading a math student on handwriting rather than on whether the answer adds up.

The researchers propose an audit and repair method. Move the critical decisions back to the AI. Grade outputs against a known correct answer instead of a format check. And report worst-case performance across repeated runs, not just the average, because an AI that scores well on average may still fail badly and unpredictably. When they applied this to one test, ComtradeBench, a leaderboard where everyone looked roughly equal turned into a clear spectrum showing which AI was genuinely reliable and which was just lucky.

So what should you do? If you are choosing an AI vendor based on leaderboard position, that ranking may not reflect how the tool performs on your actual work. The honest catch: there is no universal fix. The researchers found that whether a grader is trustworthy depends on the specific test, and scaffolding differences were uncontrolled everywhere they looked. Their conclusion is that any AI score should be read alongside how the test was built and how reliable the results were — not the headline number alone. The paper runs 31 pages with 18 figures and is posted online as a preprint, meaning it has not yet been formally reviewed by other scientists.