Misinformation Exposure in the Chinese Web: A Cross-System Evaluation of Search Engines, LLMs, and AI Overviews

A new research paper titled 'Misinformation Exposure in the Chinese Web: A Cross-System Evaluation of Search Engines, LLMs, and AI Overviews' reveals significant reliability gaps in AI-powered search tools for non-English ecosystems. Authored by Geng Liu, Junjie Mu, and colleagues, the study introduces a novel fact-checking dataset of 12,161 Chinese Yes/No questions derived from real search logs, providing the first comprehensive comparison of traditional search engines, standalone Large Language Models (LLMs), and AI-generated overview modules. The findings come as tech giants increasingly integrate LLMs like GPT-4 and Claude into search interfaces, raising critical questions about factual accuracy in global contexts beyond Western languages.

The researchers developed a unified evaluation pipeline that uncovered substantial differences in factual accuracy and topic-level variability across systems. While specific error rates weren't disclosed in the abstract, the paper indicates AI-mediated approaches showed concerning performance gaps. By combining these accuracy measurements with real-world Baidu Index statistics—which track search behavior across Chinese regions—the team estimated potential exposure to incorrect information at population scale. This methodology highlights how accuracy shortcomings in LLMs and AI overviews could systematically misinform millions of users, particularly in topics where these systems underperform. The research underscores an urgent need for more reliable, transparent information-access tools as AI becomes the primary interface for knowledge retrieval worldwide.