Fair Representation in Parliamentary Summaries: Measuring and Mitigating Inclusion Bias

A new study from researchers Eoghan Cunningham, James Cross, and Derek Greene reveals significant representational biases when large language models (LLMs) are used to summarize parliamentary proceedings. The team evaluated 5 LLMs—both proprietary and open-weight—on plenary debates from the European Parliament, developing an attribution-aware framework to measure speaker-level inclusion. Their findings show systematic biases: speakers were less accurately represented based on their speaking order (with middle speeches excluded), the language they spoke (non-English speakers disadvantaged), and their political affiliation (left-of-center parties received better representation).

The research further decomposes these biases into 'inclusion bias' (systematic omission) and 'hallucination bias' (systematic misrepresentation). Crucially, the study found that standard prompting strategies did not mitigate these issues. Instead, the researchers propose a novel hierarchical summarization method that breaks the task into extraction and aggregation steps. This approach significantly improved the speaking-order bias across all tested models, offering a practical mitigation strategy. The work underscores the urgent need for domain-specific evaluation and ethical oversight as AI systems increasingly filter and frame political content for public consumption.