Why Selfish AI Programs Starve: New Study Shows Cooperation Wins
This could shape how we build AI that works in teams instead of against us.
A team of researchers from Google and several universities built a small artificial world to answer an old question: why do living things cooperate instead of just grabbing what they want? Their world is filled with tiny computer programs that copy themselves, written in Z80 machine code (a simple, decades-old computer language). Every time a program runs, it spends energy from a shared pool. No energy, no thinking, no copying. That single rule changes everything.
The surprising result: cheating punishes itself. When energy is scarce, a program that steals from its neighbors drains the shared pool, slows everyone's processing down, and eventually blocks its own ability to reproduce. Stealing isn't just rude — it's self-defeating. Over many generations, the programs that evolved actually suppressed thievery on their own, with no referee or punishment system imposed from outside.
Location mattered too. When programs were spread out rather than mixed together, they developed richer, more complex behavior and performed better on assigned tasks. The researchers also tested simple math problems where rewards were tied to how much computing power a program had left. Cooperation kept winning. The team calls this framework 'Autopoietic Game Theory' — essentially, building social rules into the physics of the machine rather than bolting them on afterward.
Why should you care? Today's AI 'agents' (AI that can take actions on your behalf, like booking, trading, or managing resources) are increasingly being let loose in shared spaces — cloud servers, power grids, marketplaces. This paper suggests a design principle: if you tie an AI's ability to act directly to a limited, shared resource, teamwork emerges naturally. That matters as AI systems start bidding for electricity, bandwidth, or your attention. Instead of writing endless rules telling AI not to cheat, we might simply build a world where cheating runs out of fuel.
- Tiny self-copying programs were given a shared energy pool — running their code cost energy, so wasting it hurt everyone.
- Cheating backfired on its own: stealing drained the pool, slowed processing, and blocked the thief from reproducing.
- Spreading programs out led to more complex, better-performing behavior — a hint for how to design AI that cooperates.
Why It Matters
If AI is built so cooperation naturally pays off, tomorrow's assistants may be less likely to cheat or hoard.