Astronomers' AI Is Trusting the Wrong Data, Warping Galaxy Measurements

Astronomers are increasingly using artificial intelligence to analyze vast images of the sky. A new study looked at AION-1, a powerful AI trained on more than 200 million cosmic objects. The AI was designed to learn from both the raw pixels of telescope images and separate catalogues that tell it where galaxies are located. The researchers wanted to know which source the AI trusted most.

Their finding was surprising. When they kept the actual image pixels identical but changed only the detection maps — the small outlines that mark where each galaxy sits — the AI's measurements of brightness, size, shape, and distance changed by up to 4,400 times more than expected. In other words, the AI was paying far more attention to the boxes drawn around galaxies than to the light inside them. That is like a detective solving a case by reading the sticky notes on a file folder and ignoring the contents inside.

The problem is that those sticky notes are often incomplete. The survey pipeline that creates the detection maps fails to cover about 3.68% of target galaxies. Because the AI trusts these maps so heavily, that small gap causes large errors in redshift — a measure of how far away a galaxy is and how fast the universe is expanding. The researchers found that these errors could shift readings of cosmic structure by more than the acceptable threshold in 12 out of 40 test scenarios, with one extreme case exceeding it by 8.3 times.

This is a cautionary tale for AI in science. It shows that when models are trained on multiple sources of data, they may learn to rely on the easiest signal rather than the most accurate one. The good news: the problem disappears if the detection channel is simply removed or replaced with real spectroscopic data. As AI takes on more complex scientific tasks, this study reminds us to check not just what the AI learns, but what it silently ignores.