AI That Listens Can't Tell When Audio Sucks — New Test Reveals

Voice assistants, transcription tools, and other AI that listen to audio have gotten very good at understanding speech, music, and everyday sounds. But there's a simpler skill they might be missing: telling when a recording actually sounds bad. A new research project, called MRMAD, was built to test exactly this — whether these AI models can notice things like static, distortion, or a muffled microphone.

MRMAD isn't like older tests, which ask AI to identify what's being said or what's happening in an audio clip. Instead, it has a conversation-like format, where the AI hears multiple audio versions of the same sound across several rounds and has to describe how the quality changes. For example, a model might need to say which clip sounds more degraded, or what type of degradation is happening — a hiss, a drone, a crackle. This is closer to how a human sound engineer would think, and it's a much harder challenge.

The researchers ran the test on 18 different AI models, including simple ones, reasoning models, and "omni" models that handle audio, text, and images together. The results weren't encouraging: nearly all of them could understand the content — the words and sounds — but they were unreliable at diagnosing quality problems, comparing which audio was worse, or explaining what changed between clips. In other words, they hear but they don't really listen to the quality.

Why does this matter? Because real life is messy. Phone calls have echo, podcasts have hiss, concert recordings have crowd noise. If AI can't recognize that a signal is degraded, it might give you a confident but wrong transcription, or miss that a security recording has been tampered with. The MRMAD benchmark is a first step toward fixing this blind spot, so future AI can be more honest about what it's hearing — and maybe even suggest you move closer to the mic.