SidConArena Benchmark Puts LLM Agents Through Positive-Sum Negotiation

Most AI benchmarks test static reasoning or zero-sum games, but real-world economic interactions are open-ended and mixed-motive. To fill this gap, researchers from the field (Yeqi Feng, Yuxin Chen, Tianxing He) created SidConArena, a new benchmark framework that evaluates LLM agents in a positive-sum bargaining environment. The arena models a multi-player economy as a finite-horizon partially observable stochastic game with three interconnected phases: natural-language negotiation for binding trades, deterministic converter-based production, and sealed-bid auctions for long-term assets. This setup forces agents to negotiate, create surplus, compete for scarce resources, and plan under delayed returns—all while interacting via free-form natural language. The framework uses structured observations, phase-aware agent dispatching, a neural-symbolic action interface, and asynchronous execution to maintain rule-grounded evaluation without restricting agent behavior.

In both homogeneous and heterogeneous tournaments, stronger frontier models achieved higher economic outcomes, but the results exposed significant weaknesses across the board. Agents consistently misvalued resources, bargained passively rather than actively creating value, and remained severely limited in long-horizon investment planning. This suggests that even today's most advanced LLMs lack the nuanced economic reasoning and strategic foresight needed for complex, open-ended negotiations. SidConArena provides a rigorous testbed for tracking progress as models improve, and it sets a new standard for evaluating AI agents in dynamic, mixed-motive settings—a crucial step toward deploying AI in real-world economic roles like trade negotiations, supply chain management, or automated market making.