Classifying Problem and Solution Framing in Congressional Social Media
A new AI model trained on 3,467 labeled tweets can categorize political messaging with over 80% accuracy.
A team of researchers from institutions including the University of Waterloo and the University of Kentucky has published a novel study applying AI to political science. Their paper, 'Classifying Problem and Solution Framing in Congressional Social Media,' introduces an automated method to label tweets from U.S. Senators. The work is grounded in the 'Garbage Can' model of policy setting, which distinguishes between processes focused on identifying problems and those focused on proposing solutions. The researchers compiled a massive dataset of 1.68 million tweets from Senators to train their classifier.
To build the model, two academic policy experts manually labeled a subset of 3,967 tweets into three categories: 'problem,' 'solution,' or 'other.' The team then employed supervised learning techniques, rapidly iterating through models and hyperparameters. Their best-performing system was based on BERTweet Base, a variant of the BERT language model pre-trained on Twitter data. This model achieved an average weighted F1 score above 0.8 in cross-validation across the three categories, demonstrating high accuracy in distinguishing between rhetorical frames. The tool provides a scalable, data-driven lens to analyze political discourse and strategy on platforms like X (formerly Twitter).
- The model analyzed a dataset of 1.68 million tweets from U.S. Senators to classify political messaging.
- It achieved a weighted F1 score above 0.8 using a BERTweet Base model trained on 3,467 expert-labeled tweets.
- The research automates the analysis of the 'Garbage Can' policy model, distinguishing problem-focused from solution-focused communication.
Why It Matters
This provides a scalable tool for journalists, researchers, and the public to quantitatively analyze political rhetoric and legislative priorities.