Google’s AI Overviews Produce Hundreds of Millions of Inaccurate Answers Every Day, Analysis Suggests

An analysis by AI startup Oumi, conducted for The New York Times, reveals significant accuracy issues with Google's AI Overview feature. Using the SimpleQA benchmark on 4,326 searches, Oumi found AI Overviews running on Google's latest Gemini 3 model provided accurate answers 91% of the time. However, given Google's scale of over 5 trillion annual searches, even this 9% error rate translates to approximately 616 million inaccurate summaries produced every single day, assuming the feature handles half of all queries.

The problems extend beyond simple inaccuracies. Oumi's analysis found that 56% of the *accurate* answers generated by AI Overview were 'ungrounded,' meaning they cited sources that did not actually support the assertions made. This issue worsened with the newer Gemini 3 model. Furthermore, the feature frequently relied on user-generated content platforms like Facebook (cited 5-7% of the time) and Reddit as sources. Google disputed the methodology, arguing the benchmark contains flaws, but its own internal data shows Gemini 3 provides incorrect information in 28% of queries when not combined with traditional search.

This accuracy crisis has real-world consequences for information consumption. A Pew Research survey from July 2025 found users who see an AI Overview are less likely to click on traditional search result links (8% vs. 15%), and only 1% click on links within the AI summary itself. This suggests most users are not verifying the AI's claims, creating a massive potential vector for the spread of misinformation. The feature already had 2 billion monthly users as of mid-2025, operating in over 200 jurisdictions and 40 languages.