AI Brain Scans Can Give Opposite Answers, Scientists Warn

Here's a simple version of what happened. Neuroscientists often ask: what does this bit of brain activity stand for? To find out, they use AI systems — one trained on images, one trained on text — and check which one predicts the brain scan better. Whichever AI wins is said to reveal what the brain is 'representing.' It sounds tidy. But this new paper shows it isn't.

The team used brain scans of people looking at photographs, along with DINOv2 (an AI that turns images into numbers) and MPNet (an AI that turns sentences into numbers). They compared whether short photo captions or longer spoken-style narratives matched the brain better. When they compared one short caption against a long narrative, language 'won.' But once they matched the two for rough word count, the result flipped — and vision won instead. The flip showed up in every brain region they measured, in both volunteers.

So what's the takeaway? A head-to-head race between two AI models is not a clean window into the brain. If one model simply gets more text or different text, it may win for boring reasons — not because the brain cares about that content. The authors argue that 'predictor construction' — how you build the AI side of the experiment — is part of the experiment itself, not a neutral setup detail.

Why should a normal person care? Because headlines about AI reading minds, decoding dreams, or mapping thoughts often rest on exactly these comparisons. This paper is a quiet warning that some of those claims may be shakier than they sound. The practical fix is matched controls: compare like with like. That's better science, and it means fewer breathless stories about machines that 'understand' your brain.