Want AI to Judge Photos Better? Ask for Details, Not Opinions

AI models can now look at a photo and give it a beauty rating, and the usual advice is to use a panel of several models — like a judging panel at a talent show. Researchers tested that idea on two sets of images that real people had already rated. Their finding: a panel of overall verdicts never clearly beat its single best member, whether you averaged the scores or used a smarter combining method.

The fix turned out to be about what you feed the panel. Instead of asking "how good is this photo?", they had each AI score five specific dimensions from a fixed, human-written checklist. Those detailed scores really did measure what they claimed — in 28 of 30 tests, a dimension question revealed more specific information than a general one. When combined, the detailed scores beat the best single AI in all ten three-model panels they tried. It wasn't just because there were more numbers: repeating general scores the same number of times didn't produce the same gain.

The catch is cost. This approach used about 4.8 times as many AI requests, plus a few hundred labeled examples that had to be made fresh for each dataset — they didn't carry over. On one dataset, a single model already captured 85% of the human ceiling, meaning there was little room left to improve. The authors also honestly report a failed pre-registration and the setups that lost.

Why should you care? Every time a photo app, stock image site, dating app, or ad platform picks the "best" picture automatically, an AI judge is doing the picking. If those judges understand the specific reasons a photo works — not just a vague thumbs up or down — the images you see, and the ones of you that get chosen, could get noticeably better.