Scientists Figure Out How AI Sees Color — And Built a Tool

When an AI looks at a photo, it doesn't "see" a red apple the way you do. It turns the picture into long lists of numbers, called a representation (the AI's internal notes about an image). Nobody fully understood how color survives that conversion. This paper, from researcher Yuan Sun, works out the rules — when color information is kept, when it gets lost, and why.

The surprising part is how tidy it is. When the AI looks at a color wheel, the color information doesn't smear randomly across its notes. Instead, 84 to 88 percent of it squeezes into just the first two "harmonics" — think of these as the two simplest tones in a piece of music. Better still, the same two rotation planes show up across different shapes: circles, squares, triangles. That structure is partly inherited from the input and the AI's architecture, and partly reshaped by training and by how deep the network is.

Guided by those measurements, Sun built a compact interface whose color behavior is set by math, not learned from data. It was never fitted or trained on color. Yet it reads hue — the position on the color wheel — with a median error of just 3.4 degrees on shapes it had never encountered. For comparison, most people can't reliably tell two colors apart when they're less than a few degrees apart, so this is effectively human-level accuracy, for free.

The bigger idea is that understanding how an AI represents something can let you design capabilities directly instead of paying for them with huge training runs. That could mean cheaper, faster, more predictable image tools. The honest catch: this is one 46-page preprint focused entirely on color, a narrow and unusually clean case. Whether the same trick works for faces, text, or medical scans is unknown, and nothing here is a shipping product.