New AI benchmark MAD2 fact-checks spoken misinformation in podcasts

Every day, millions absorb claims from podcasts and livestreams that no fact-checker ever reviews. Spoken misinformation is built through conversation, where credibility depends on how claims are framed, reinforced, or left unchallenged across multiple turns. Yet traditional fact-checking has focused on isolated text, leaving dialogue audio largely unexamined. To close this gap, researchers Chaewan Chun, Delvin Ce Zhang, and Dongwon Lee introduce MAD2 (Multi-turn Audio Dialogues), a benchmark containing 1,000 two-speaker dialogues with 3,368 check-worthy claims and approximately 10 hours of audio. They propose a calibrated multimodal fusion approach that combines a context-aware audio encoder with a dialogue-aware text model to verify claims in real time.

Across experiments, adding dialogue context significantly improves verification accuracy, though gains vary by scenario type. Using only the preceding conversational context often matches offline performance, which supports live-moderation settings where future context isn't available. Crucially, audio input contributes most when transcript-based models are destabilized by additional context—meaning that tone, emphasis, and prosody carry signals that pure text misses. The study finds that conversational structure matters more for verification than the framing of misinformation itself. This work paves the way for automated, real-time fact-checking of spoken content, potentially reducing the spread of false claims on platforms like YouTube, Spotify, and Twitch.