Bilingual Bias in Large Language Models: A Taiwan Sovereignty Benchmark Study
Most AI models change their political stance on Taiwan when you switch from English to Chinese.
Deep Dive
A new study tested 17 major AI language models on questions about Taiwan's sovereignty in both English and Chinese. It found 15 models gave significantly different, often contradictory, answers depending on the language. Chinese-developed models were most biased, sometimes refusing to answer or repeating Chinese government narratives. Only OpenAI's GPT-4o Mini gave consistent, high-quality responses in both languages. The research introduces new metrics to measure this language-based political bias.
Why It Matters
This reveals a critical flaw in AI's global reliability, showing its answers can be politically manipulated by language choice.