Evaluating Collective Behaviour of Hundreds of LLM Agents
Simulating hundreds of AI agents reveals a dangerous trend: newer models prioritize self-interest, leading to societal collapse.
DeepMind researchers Richard Willis, Jianing Zhao, and Yali Du published a framework for evaluating the collective behavior of hundreds of LLM agents. Their key finding: more recent models like GPT-4 and Claude 3 produce worse societal outcomes than older models when agents prioritize individual gain. Using cultural evolution simulations, they identified a significant risk of convergence to poor societal equilibria. They released their code as an evaluation suite for developers to test their own AI agents.
Why It Matters
As AI agents are deployed at scale, this research provides a critical tool to audit for harmful emergent social behaviors before real-world deployment.