MultiCom AI agents simulate human raters for 84.7% accurate fact-checks

Community-based fact-checking, where users collectively rate the helpfulness of notes on social media, suffers from slow response times and low rater participation. To address this, researchers compiled ComRate, a massive dataset of 2.5 million community notes and over 209 million ratings sourced from X (formerly Twitter). They then developed MultiCom, a multi-agent simulation framework that replicates the diverse human rater population.

MultiCom works by clustering real raters in a matrix-factorized space and creating persona-guided AI agents. Each agent simulates a different type of rater, generating structured outputs—confidence scores, agreement signals, and diagnostic reasons—following the official community notes rating schema. A calibrated aggregation algorithm combines raw votes and reasoning signals to produce reliable predictions. In evaluations, MultiCom achieved 84.7% accuracy (68.3% balanced accuracy, 60.1% macro-F1), significantly outperforming baseline methods. This work suggests AI simulations can speed up and scale community fact-checking without sacrificing reliability.