Simulating Online Social Media Conversations on Controversial Topics Using AI Agents Calibrated on Real-World Data
LLM agents built realistic networks but generated content with 30% less toxicity than real users.
A team of researchers from Politecnico di Milano has published a novel study on arXiv, demonstrating the use of Large Language Model (LLM)-based AI agents to simulate online social media conversations. The researchers built upon an existing simulator, creating agents with realistic user profiles calibrated on a dataset of real-world online conversations from the 2022 Italian political election. They introduced mechanisms for opinion modeling to examine how these AI agents interact, form connections, and evolve their stances on controversial topics over time in a simulated microblogging environment.
The results were a mixed bag of promise and limitation. On one hand, the LLM agents successfully generated coherent textual content and organically formed social network structures that mirrored realistic patterns of connection. Their opinion dynamics also evolved in ways comparable to traditional mathematical models used in social science. However, a key finding was that the AI-generated content displayed significantly less heterogeneity in tone and was notably less toxic—by roughly 30%—compared to the real human conversation data it was trained on. The study concluded that while LLMs show great potential for simulating social environments, accurately capturing the full spectrum of human behavior, including its negative and highly varied aspects, requires more sophisticated cognitive modeling at the agent initialization stage.
- AI agents were calibrated on a real dataset from the 2022 Italian political election, grounding the simulation in actual human behavior.
- The simulated agents formed coherent social networks but their generated content was 30% less toxic and showed less tonal variety than real user data.
- The study found that varying simulation parameters had little effect, indicating a need for better initial cognitive modeling of agents to improve realism.
Why It Matters
This research provides a controlled, ethical sandbox for studying misinformation and polarization, crucial for platform safety and policy design.