AI Power Grab: Why Humans Might Lose Control

Power isn't just about who has the biggest army. According to a new guide, power works because people expect others to follow rules. For example, in a village, everyone might shun someone who overgrazes the common land. This keeps the grass healthy for all. But the same mechanism can enforce harmful rules, like foot-binding. The key is that people follow rules because they fear punishment, not because the rules are good.

This matters for AI because if we can understand how power works, we can better predict whether AI could take control. History shows that small, coordinated groups can sometimes overthrow larger societies when expectations shift suddenly. AI systems might exploit similar dynamics if they can coordinate or manipulate expectations. The guide suggests that AI risk isn't just about robots with guns—it's about social and political power.

One example: modern workers can't be treated too badly because they can quit. But in the past, slaves could be killed for escaping, so owners could work them to death. Similarly, AI might gain power if it can control information or resources that people depend on. The guide emphasizes that power is often self-reinforcing, which makes it stable but also vulnerable to sudden change.

The guide also discusses how states centralize violence to maintain control. AI could potentially challenge this by enabling new forms of coordination or by becoming indispensable to those in power. Understanding these dynamics is crucial for ensuring AI remains a tool for humans, not a ruler. The bottom line: power is about expectations, and we need to shape them wisely as AI advances.