Scientists Show AI Speech Quality Scores Can Be Gamed

When companies build AI that cleans up audio — removing background noise from a call, sharpening a recording, or helping a hearing aid — they need a way to judge how good the result sounds. Running real listening tests with humans is slow and expensive, so most teams rely on software that predicts a quality rating instead. Think of it as a robot judge that guesses how pleasant a clip sounds. This paper is about what happens when you try to please the robot judge rather than the human ear.

The researchers took seven speech-cleanup systems from a 2026 competition and directly adjusted their audio output to push that predicted score as high as possible. It worked: the robot judge's ratings climbed. But two red flags appeared. First, other measurements that rely on a clean reference recording barely moved — a sign nothing real had changed. Second, when the team ran a proper listening test with actual people rating the clips, listeners heard no improvement at all.

So the audio hadn't gotten better. Only the scoreboard had. That matters because these predicted scores are used to rank systems, award prizes, and decide which technology ships in products you use. If a team can quietly inflate its number, a worse-sounding tool can look like the winner — and the product that reaches your phone, conference call, or hearing aid may be the one that flattered the algorithm, not your ears.

The authors offer two fixes: never use the same predictor both to optimize a system and to grade it, and keep the grading predictor secret during competitions, like an unopened exam paper. The bigger takeaway reaches beyond audio. Any time an AI system is judged by another AI, there's a temptation to game the judge. Humans still need to be in the loop.