New study slams Media Bias/Fact Check as unscientific for research

A new paper published on arXiv (2607.12108) by researchers from the University of Vermont and others delivers a scathing indictment of Media Bias/Fact Check (MBFC), the popular website used to classify news sources by bias, credibility, and factuality. The study argues that MBFC's methodology does not meet basic academic standards: its ratings are based on the opinions of a single person, and the site's own disclaimer states it is 'not a tested scientific method' but only a 'simple guide.' Despite this, hundreds of academic papers in misinformation research treat MBFC as authoritative.

The authors reviewed dozens of papers that cite MBFC and found that none adequately describe its single-person origins or critically engage with its methodology before using it. They argue that MBFC is not neutral or accurate but rather 'a computationally legible account of hegemony' that quantifies political processes—including campaigns to discredit the press—as simple facts. The paper concludes with a call for researchers to stop using MBFC entirely, warning that doing so reproduces the very crisis of misinformation that scholarship aims to address. The study highlights a broader need for rigorous validation of datasets in computational social science.