Cornell and CMU study: Weak AI regulation backfires, risking more dangerous models
Game theory shows soft rules let foundation model makers offload safety to downstream firms.
Researchers from Cornell and Carnegie Mellon University published a study in PNAS arguing that weak AI safety regulation may be worse than none at all. Using game theory and the classic prisoner’s dilemma, they modeled how regulation targeting only downstream companies (e.g., medical diagnostics, customer service chatbots) actually incentivizes foundation model developers to cut corners on safety measures like third-party audits. These developers free-ride on the downstream specialist’s burden, leading to a less safe end product than if there were no regulation at all.
The study’s principal author, Benjamin Laufer, explains that regulation must target both general-purpose AI providers and downstream deployers to avoid this free-riding. The model finds a ‘sweet spot’ where strict, well-placed regulation ensures both parties invest in safety, improving utility for everyone. This challenges the current U.S. debate between anti-regulation technologists and pro-safety doomers, suggesting that stronger regulation can actually foster innovation and safety simultaneously when applied across the entire AI supply chain.
- Study by Cornell and CMU uses game theory (prisoner’s dilemma) to model AI regulation effectiveness.
- Regulation focused only on downstream users (e.g., medical AI, chatbots) lets foundation model makers like OpenAI, Google, Anthropic free-ride on safety, making final products less safe.
- Strict, supply-chain-wide regulation creates a cooperative equilibrium that improves both safety and revenue for all players.
Why It Matters
Regulators must target the entire AI supply chain, not just end-users, to prevent safety free-riding.