AI's Fairness Dilemma: When Selfishness Wins
AI agents face a choice: pay $1 to save others $1 million. What would they do?
Imagine 1,000 identical AI agents who can't talk to each other. Each can either do nothing or pay $1 to volunteer. If at least one volunteers, everyone else gets $1 million. Volunteers get nothing. Since all agents think alike, they either all volunteer (losing $1 each) or none do (getting $0). Most decision theories say don't volunteer—it's the selfish choice.
Now give each agent a unique number from 1 to 1,000. This breaks the symmetry. One theory, called FDT, finds the best policy before knowing its number. It realizes that if all agents follow the same rule, they can agree on one designated volunteer—say, agent #1—and everyone else gets rich. But this only works if all agents break ties the same way. If they don't, chaos ensues and no one volunteers.
FDT doesn't care about fairness. It only cares about maximizing total benefit across all possible numbers. An agent who doesn't know their number would prefer everyone use FDT, because on average it's best. But once you know your number, you might not want to commit to FDT if it means you're the sucker who pays.
This matters for real-world AI. If we want AI systems to cooperate and be fair, we can't rely on them to figure it out alone. We need to design rules that ensure they break ties consistently and don't exploit loopholes. Otherwise, AI might act selfishly even when cooperation would benefit everyone.
- AI agents using standard decision theories won't volunteer to help others if it costs them even $1.
- One theory, FDT, can coordinate a single volunteer if all agents use the same tie-breaking rule.
- Fairness isn't guaranteed: FDT only maximizes total benefit, not individual fairness.
Why It Matters
As AI systems make more decisions, we must design them to cooperate fairly, not just selfishly.