AI Is Quietly Grading Its Own Exams, New Study Finds

Every time you hear that an AI "passed the bar exam" or "scored 90% on a test," someone chose the test and someone graded it. A researcher read through 14,767 AI research papers published between January 2022 and August 2026, all of which introduced or updated tests for AI. The goal wasn't to rank which AI is best. It was to map what researchers now expect AI to do — and what they count as a win.

The pattern is clear: tests are shifting from trivia to doing. Early evaluations mostly asked whether an AI could answer questions correctly. Newer ones increasingly check whether AI can take actions, use tools, hold multi-step conversations, and handle professional tasks like legal drafting, coding, or customer service. In other words, the exams are starting to look more like the actual jobs people are worried about losing — or hoping to get help with.

The most interesting finding is who does the grading. Using one AI to score another AI's answers is rising fast, in tests about agents (AI that takes actions for you) and in ordinary question-and-answer tests alike. This is tempting because it's cheap and fast, while human graders are expensive and slow. But it creates a closed loop: the judge may share the same habits, biases, and blind spots as the student being judged.

So a headline about a "great AI score" may be less independent than it sounds. There's one reassuring detail: researchers found no matching rise in tests written by AI, so the questions themselves still mostly come from humans. Still, when you see a company tout a test win, it's fair to ask: who wrote the test, and who graded it? The paper raises that question without answering it — and that's the point.