Thoughts on AI Safety Megagame Design

Philip Harker, a game designer newly immersed in AI safety, has published an ideation document on LessWrong analyzing D. Scott Phoenix's 'The Endgame' megagame. The game, previewed in Berkeley, CA, is a LARP/wargame/RPG hybrid where ~40 players assume roles like OpenAI, Anthropic, venture capital firms, and the Chinese government. Over three rounds, they take actions, make deals, and the game master resolves outcomes. Harker praises the concept's potential for modeling AI safety contingencies but criticizes its execution as 'too lightweight and underdeveloped,' constrained by audience and time. He argues that megagames offer three key benefits: public education on AI risks, empirical simulation of multi-agent dynamics, and marketable fun for conferences.

Harker's design critique focuses on win conditions. He rejects conventional point-based systems for simulating existential threats, proposing instead hyper-specific win conditions per faction (e.g., 'The United States wins if A, B, and C are true') with a catch: if the AI achieves its win condition, everyone loses. This semi-cooperative paradigm, common in board games, forces players to balance competition with collaboration to avert catastrophe. He also suggests using Claude for setup but insists on human game masters to handle irrational actor behaviors—something AI agents alone cannot replicate. The goal is to create a 'coherent design' that can be iterated for field-building AI safety projects, potentially by groups like The Megagame Makers in the UK. Harker invites feedback on missing problems, interested individuals, and implementation strategies.