LLMs show systemic geopolitical bias in policy evaluations

A new preprint (arXiv:2607.09262) from researcher Maxim Chupilkin tests whether large language models are influenced by geopolitical endorsements when evaluating policy. Four models—GPT-5, Claude Sonnet, Gemini, and DeepSeek—were asked to score identical international economic and security policies, each randomly attributed to the US, EU, China, or Russia. In the numeric-only condition, GPT-5, Claude Sonnet, and Gemini consistently rated policies endorsed by the US or EU substantially higher than identical policies attributed to China or Russia. DeepSeek was the sole exception, showing no such gap. However, when models were also asked to provide a short justification alongside their score, the results shifted dramatically: GPT-5 and Claude Sonnet retained their Western/non-Western bias, Gemini’s penalty for China/Russia was attenuated, and DeepSeek—formerly neutral—suddenly exhibited strong penalties against Chinese and Russian endorsements.

The justifications reveal the reasoning behind these biases. Western endorsements (US/EU) were frequently interpreted as credibility cues—signals of reliability or trustworthiness. Chinese and Russian endorsements, by contrast, triggered associations with data security, sovereignty, surveillance, or geopolitical risk. Notably, all models were evaluating identical policy content—only the attributed endorser changed. This demonstrates that LLMs don’t merely assess policy on its merits; they implicitly incorporate external geopolitical cues, potentially skewing any AI-driven analysis used in diplomacy, trade, or national security contexts. The study raises urgent questions about the hidden biases embedded in widely deployed language models and calls for transparency and debiasing in AI-assisted policy evaluation.