Squeezing OpenAI's Whisper AI Adds Hidden Errors for Some Accents

Speech-to-text tools like OpenAI's Whisper are how your phone turns voicemails into text, how meeting apps write notes, and how call centers scan conversations. Companies rarely run the full-size model. They shrink it — trimming away parts and storing numbers with less precision — so it runs fast and cheap on a phone. This study asks a simple but rarely asked question: does that shrinking make the tool worse for some voices than others?

The answer is yes. The team tested the Whisper family on three voice datasets. After trimming half the model (a technique called pruning), the accuracy gap between the best-served and worst-served groups more than doubled. At a modest estimate of five seconds to fix each mistake, correction time rose from 30 to 64 seconds per minute of speech — a 111% jump. That is over an hour of fixing for every hour of audio. The pattern held even after the researchers controlled for audio quality.

The worst case involved extreme compression for small devices. When weights were squeezed down to four bits (called INT4 HQQ quantization — think saving a photo at very low quality), West African accents triggered "catastrophic loops," where the transcript repeats itself endlessly, five to seven times worse. Not all shrinking is harmful: distillation (training a small model to copy a big one) actually narrowed the gap in 21 of 27 tests.

The takeaway: fairness checks run on full-size models do not tell you what the version in your app actually does. The researchers call the extra effort "temporal taxation" — a time tax paid by people whose speech the tool handles poorly. If you build or buy speech tools, or you rely on auto-captions, test the compressed version. Cheap AI is not equally cheap for everyone.