Sustaining Cooperation in Populations Guided by AI: A Folk Theorem for LLMs

A new paper from computer scientists at Bar-Ilan University and Tel Aviv University establishes a 'folk theorem for large language models (LLMs),' demonstrating that when multiple LLMs each advise a population of clients playing repeated games, cooperation can emerge and persist even when the underlying incentives are misaligned. The authors—Jonathan Shaki, Eden Hartman, Sarit Kraus, and Yonatan Aumann—model a meta-game in which LLMs compete to guide clients, creating indirect strategic interaction. Their key finding: in repeated settings, all feasible and individually rational outcomes can be sustained as ε-equilibria, despite clients not knowing which LLM advised their opponents. This extends classic folk theorem results to scenarios with limited observability and agent autonomy.

The paper, submitted to arXiv on May 7, 2026, also analyzes one-shot interactions, where cooperation is possible only when a single LLM can influence multiple roles in the same game. In repeated games, the shared reliance on a common LLM effectively couples independent agents, enabling sustained cooperation through the threat of future punishment. The proof requires novel techniques beyond the standard folk theorem. This work has implications for multi-agent AI systems, autonomous trading, and any domain where LLMs serve as advisors to collectives—suggesting that AI alignment need not only be enforced per-agent, but can also emerge from the strategic interplay of the LLMs themselves.