AI Research Helpers Often Cite the Wrong Page, Study Finds

A researcher built an AI system for a competition where the job is simple to describe: someone asks a research question, and the AI must find the right papers, point to the exact page, table, or figure where the answer lives, and then answer in the requested format. The test pool held 27,487 scientific papers. The goal sounds like what every AI research assistant promises you today — not just an answer, but proof of where the answer came from.

The study's big finding flips the usual assumption. The AI wasn't bad at reading. It was bad at finding. When the system handed the AI the correct page, it pointed to the right spot 45 out of 48 times — about 94% accuracy. But the page-picking step delivered the correct page only 52.6% of the time. In other words, the reading half of the job was nearly solved; the looking-up half was a coin flip. That matters because most people assume the hard part is comprehension, when the real weakness is search.

The most worrying detail is how the system failed. Out of 45 cases where the right page was never shown, it returned nothing 14 times, named the wrong page 24 times, and stumbled onto a correct page 7 times. So it failed quietly — with a confident, wrong citation — nearly twice as often as it failed visibly by staying silent. For anyone reading an AI-written summary of medical research, a legal case, or a market report, a wrong citation that looks right is the dangerous kind of mistake.

The fix is almost embarrassingly simple: stop choosing pages at all. Every retrieved paper fit inside the amount of text the AI can hold at once, so the team showed it whole papers instead of a pre-selected page. Correct-page coverage then hit 100% on the papers they could process. The system finished with 0.762 paper accuracy, 0.441 evidence accuracy, and 0.920 on multiple-choice questions. The takeaway for you: when an AI tool cites its sources, check whether it actually read the source — or just guessed where to look.