New AI Voice Systems Struggle with Accents—Here’s Why

A new AI research paper reveals a frustrating problem for anyone who speaks with an accent: voice assistants are getting worse at understanding you, even when they try to look up answers. The study tested AI systems that answer questions by first turning your voice into text (like Siri or Alexa), then searching for the answer online. These systems use a technique called multi-hop RAG (RAG, or retrieval-augmented generation, lets AI look things up to improve answers).

The researchers found that when someone speaks with an accent, the AI’s first step—turning speech into text—often makes small mistakes, like mishearing a name or word. Normally, you’d expect the AI to catch and correct these errors by double-checking its work. But the study discovered the opposite: the fancier the AI’s lookup system, the more it amplifies these tiny mistakes. For example, the AI might mishear “Paris” as “purr-is,” then look up the wrong thing and give you a completely unrelated answer.

The problem isn’t just about accents—it affects anyone whose voice isn’t perfectly clear to the AI. The researchers tested four different AI setups and found that the smarter the system, the bigger the gap between correct answers and wrong ones. The most common issue? The AI messes up the names or key details in your question, and those errors snowball as the system tries to “improve” its answer.

So what can be done? The study tested simple fixes, like rewriting the question to sound more “standard,” but these didn’t help much. The researchers say the issue is in how the AI’s lookup system is designed—not just the first step of turning speech into text.