Study shows content moderation’s aggregate scores hide harm to bridge users

A new arXiv paper by Igor Itkin reveals a dangerous blind spot in AI-driven content moderation: systems can ace every aggregate accuracy metric while systematically harming the few users who hold communities together. Using an agent-based model with 240 learning agents arranged in a community-structured network, the simulation tasked a noisy classifier with flagging harmful posts while a regulator penalized or removed flagged content. The overall utility of the network barely budged across different noise profiles (one-way ANOVA, p=0.96), suggesting that by standard measures, nothing was wrong.

But the damage was concentrated entirely on bridge users—the small number of individuals who connect otherwise separate communities. For these users, the noisy classifier both suppressed valuable content and failed to catch dangerous posts. Itkin introduces a new governance loss metric (L_gov) that separately prices these two error types, and shows that under false-positive-heavy noise, L_gov more than doubles. The paper also finds that a user’s degree (how many connections they have) serves as a near-perfect proxy for the betweenness that defines a bridge (r=0.96), making it a cheap and effective audit target. The implication: platforms must look beyond aggregate accuracy and monitor how errors are distributed across network roles.