Researchers propose MECV to slash AI-generated news hallucinations
New framework improves factual reliability in AI news summaries by 40% using multi-LLM consensus and cross-platform verification.
Researchers Zhuo Xie and Haoze Ni have introduced the **Multi-source Evidence Consensus Verification (MECV)**, a post-hoc hallucination correction framework designed to improve the factual reliability of AI-generated news summarization. Unlike traditional methods that rely on single retrieval channels, MECV aggregates evidence from multiple heterogeneous sources, including the original document, Wikipedia, and open-web retrieval. This cross-platform approach is augmented by a multi-LLM jury mechanism that estimates factual reliability through contradiction-aware consensus scoring across verifier models like GPT-4o-mini and DeepSeek-Chat.
The framework was evaluated on the SummEdits benchmark using Qwen-Plus as the orchestrator, demonstrating a significant improvement in factual consistency while maintaining the semantic structure of the original summaries. Notably, MECVβs use of agreement across heterogeneous evidence sources proved particularly effective in identifying factual uncertainty, even in information-sensitive domains such as financial news aggregation. This work contributes to the broader goal of trustworthy AI and automated journalism by introducing a robust, consensus-based verification system for mitigating hallucinations in AI-generated content.
- MECV aggregates evidence from multiple sources (source document, Wikipedia, open-web) and uses a multi-LLM jury (GPT-4o-mini, DeepSeek-Chat) for contradiction-aware consensus scoring.
- Evaluated on SummEdits with Qwen-Plus, MECV improves factual consistency while preserving semantic structure, including in sensitive domains like financial news.
- The framework identifies factual uncertainty by leveraging agreement across heterogeneous evidence sources, reducing hallucinations in AI-generated news summarization.
Why It Matters
This framework could revolutionize automated journalism by making AI-generated news summaries more trustworthy and reliable for critical applications.