LLMs outperform sentiment analysis for political news
New research shows LLMs detect bias and framing where traditional NLP fails in 70% of cases
Researchers Maryam Fooladi and Federico Bottino have published a groundbreaking study comparing traditional NLP sentiment analysis with modern LLM-based approaches for evaluating political news coverage.
The team analyzed 50 political news articles from 17 international media outlets using two methods: a RoBERTa-based sentiment classifier and a custom LLM framework designed for multi-dimensional analysis. The results revealed a critical flaw in traditional sentiment analysis, which they term "neutral collapse" - with 70% of articles being classified as neutral despite containing rich political content. Even more concerning, 23% of these neutral-classified articles showed negative probability scores above 0.30, indicating misclassified bias. The LLM-based approach successfully detected political bias direction and intensity, sensationalism levels, emotional appeal patterns, and framing techniques - dimensions essential for social science and humanities research but completely missed by traditional sentiment analysis.
- RoBERTa misclassified 70% of political news articles as neutral, hiding critical bias information
- LLM-based multi-dimensional analysis detected political framing, sensationalism, and emotional appeals in all 50 articles from 17 outlets
- Study authors argue LLM frameworks better align with social science research needs than traditional sentiment analysis
Why It Matters
Professionals in media analysis and political science can now get nuanced political bias insights instead of oversimplified sentiment scores