MIT uses AI agents to audit X’s algorithm at scale

Researchers from MIT (Alessandro Morosini, Sarah H. Cen, Andrew Ilyas, Hedi Driss, Aleksander Mądry, Chara Podimata) introduced a framework that uses generative AI agents as behavioral engines for synthetic social media accounts. Each agent is instantiated with a fixed persona grounded in demographic and political survey data, then interacts with platform content by reasoning and choosing actions. Because behavior is fixed within each persona while platform‐visible signals (age, gender, location) can be experimentally perturbed, the design enables counterfactual auditing — measuring how algorithms respond to user attributes independently of behavior. This overcomes the tradeoff between costly real‐user studies and unrealistic scripted sock‑puppet audits.

As a case study, the team deployed 1,120 agents on X shortly after the 2024 U.S. election, spanning 14 personas and three counterfactual conditions. Over 200,000 content exposures were collected. Analysis revealed that X’s algorithmic feed amplifies toxic, polarizing, political, and right‑leaning content relative to the chronological feed, with amplification varying sharply by user ideology. Counterfactual experiments showed that demographic signals affect content delivery in persona‑dependent ways: pooled effects were largely null, but subgroup‑level effects varied in direction and magnitude. This work establishes GenAI‑based agents as a scalable, realistic tool for algorithmic auditing, with implications for transparency and regulation of online platforms.