Debiasing International Attitudes: LLM Agents for Simulating US-China Perception Changes
Researchers used GPT-4o and Qwen3-14b to simulate 20 years of opinion change, revealing inherent model biases.
A team of researchers, primarily from Tsinghua University, has published a novel computational social science study that uses Large Language Model (LLM) agents to simulate and analyze the evolution of U.S. public opinion toward China over a 20-year period (2005-2025). The framework integrates large-scale news data and social media profiles to initialize populations of AI agents, which then undergo cognitive-aware reflection and opinion updating when exposed to information. The core innovation is the testing of three distinct debiasing mechanisms designed to produce more objective, human-like opinions from the AI agents.
The three debiasing methods tested were: fact elicitation (extracting neutral events from subjectively framed news), a 'devil's advocate' agent that provides critical counter-arguments, and counterfactual exposure to surface the models' inherent biases. Simulations were run using two state-of-the-art LLMs: OpenAI's GPT-4o and Alibaba's Qwen3-14b. While exposure to media led to the expected negative attitudinal trend, the 'devil's advocate' agent proved most effective at mitigating this bias, suggesting intermediate analytical steps are key. A critical finding was that the counterfactual study revealed contradictory results between the two models, indicating strong, region-specific inherent biases—likely tied to the geographic and cultural origins of their training data.
This research represents a significant step in using AI agents for complex social simulation. It moves beyond simple sentiment analysis to model dynamic opinion formation, providing a new tool for understanding global polarization drivers like media influence. The study's conclusions highlight both the promise of LLMs for social science and a major caution: the models themselves are not neutral platforms but carry embedded perspectives that must be accounted for in any simulation of human attitudes.
- The study built an LLM-agent framework simulating 20 years of U.S. attitudes toward China using GPT-4o and Qwen3-14b.
- A 'devil's advocate' agent was the most effective of three debiasing methods, reducing negative attitudinal trends from media exposure.
- Counterfactual tests revealed contradictory findings between models, exposing region-specific inherent biases tied to their training origins.
Why It Matters
Provides a new AI-powered tool for modeling polarization and reveals critical inherent biases in LLMs that affect social science simulations.