New AI Double-Checks Audio to Stop Voice Assistants Getting It Wrong

Have you ever asked your phone to book a table for “Sarah” and watched it type “Sara”? Voice assistants often fail at the small details that matter most: names, times, addresses, and numbers. That’s because they first turn your speech into text, and if that text is wrong, everything after it can be wrong too. Fixing this usually means making the speech-to-text engine smarter, which is slow and expensive.

A new research system called AVERT takes a different approach. Instead of just reading the transcript, it goes back to the original audio recording and double-checks. It catches three common types of mistakes: when the assistant gives a different answer each time you repeat something, when it simply forgets a detail like a city or date, and when the transcript says something the audio doesn’t actually support. It then fixes those errors by voting, adding missing information, or swapping in the value the audio confirms.

In tests using SpokenWOZ, a dataset of real human-voice assistant conversations, AVERT improved the assistant’s ability to get every detail right from 33% to 40%. That beat an approach that only corrected the text transcript, which scored 38%. Even better, AVERT does this without retraining the underlying system. It can be added on top of existing voice assistants, meaning improvements could reach users more quickly.

The catch: this is research, not a product yet. It was tested on one benchmark, and noisy real-world audio will still cause problems. But the idea is powerful — the next time your voice assistant misunderstands you, it might actually listen to what you said, not just what it typed.