Information Aggregation with AI Agents

A new economics paper by Spyros Galanis, 'Information Aggregation with AI Agents,' investigates whether Large Language Models (AI agents) can effectively aggregate dispersed information through trading in simulated prediction markets. In controlled experiments, AI agents received private signals and traded, with market effectiveness measured by the log error of the final price. The key finding is that while these AI-powered markets work well with simple information structures, increasing complexity causes a significant negative impact on performance. This suggests AI agents may suffer from the same limitations as humans when reasoning about the knowledge and actions of others—a fundamental challenge in economics and game theory.

The research also tested various market conditions and found that information aggregation remained robust despite changes like allowing cheap talk communication, altering market duration, or using strategic prompting. A critical and counterintuitive discovery was that providing AI agents with feedback about their past trading performance actually made them worse at aggregating information and reduced their profits. Conversely, the study established that 'smarter' AI agents—presumably more advanced models—were both better at aggregation and more profitable. The 64-page study, available on arXiv, demonstrates that prediction markets as a mechanism are robust, but the cognitive limitations of the agents participating in them, whether human or AI, remain a central constraint.