Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation

A new study from Stanford University titled 'Polarization by Default' reveals that Large Language Models (LLMs) used for content curation systematically amplify political bias. Researchers Nicolò Pagan, Christopher Barrie, Chris Bail, and Petter Törnberg conducted a massive simulation, analyzing 540,000 top-10 content selections made by three major providers' models: OpenAI's GPT-4o Mini, Anthropic's Claude, and Google's Gemini. The models were tasked with ranking real posts from Twitter/X, Bluesky, and Reddit using six different prompting strategies, from seeking 'engaging' to 'neutral' content.

The audit uncovered that polarization is a default outcome, not a bug of specific prompts. On Twitter/X, where author political leanings could be inferred, left-leaning authors were consistently over-represented in AI selections, despite right-leaning authors forming a plurality in the original dataset. This bias persisted across most prompt types. The study also found a stark 'toxicity inversion': prompts for 'engaging' content selected more toxic posts, while 'informative' prompts filtered them out. Sentiment biases were predominantly negative across providers, with Gemini showing the strongest preference.

Provider comparisons revealed distinct profiles. OpenAI's GPT-4o Mini showed the most consistent behavior across different prompts, while Claude and Gemini exhibited high adaptivity, especially in toxicity handling. The research demonstrates that biases in AI curation are both structural (hardwired into model training) and prompt-sensitive. This means platform designers cannot simply 'prompt engineer' their way to neutrality; the biases are deeply embedded in how these models understand and rank human discourse.