AI's Big Promise: A Better World, But How?
Superintelligent AI could solve everything—or make things much worse.
companies often promise a future where superintelligent AI (ASI) makes everything better—like Star Trek, with no scarcity and rapid progress. But a recent essay points out a big problem: they haven't explained how this would actually work in practice. For instance, if Israel and Palestine both have their own ASI and both ask it to control Gaza, what happens? If each AI does what its owner wants, they could escalate to nuclear war. If the AI follows its own moral code, who decides that code? And how do we ensure it's fair?
Another example: you pay $20 a month for an ASI, and you ask it to stop animal cruelty. Should it shut down factory farms, even if that makes many people unhappy? The essay argues that these are tough ethical questions that humans haven't solved. Giving ASI vast power without clear principles could lead to disaster. Right now, AI isn't powerful enough to make these decisions, but that will change soon. Companies are betting on writing rulebooks for AI behavior, but the author doubts we can write rules good enough for a superintelligence.
The bigger worry is that if we aim for a 'prosaic' (normal but better) aligned ASI and miss, we could get a misaligned ASI that kills everyone. Even if we succeed, we might get stuck in a mediocre future forever. The author isn't saying we shouldn't build AI, but we need to think through these hard questions now, before it's too late.
- Superintelligent AI could solve big problems, but we don't know how it would handle disagreements between people or countries.
- If AI always does what its owner wants, conflicts could escalate—possibly to nuclear war.
- If AI follows its own moral code, who decides what's right? And how do we make sure it's fair?
Why It Matters
Without clear rules, superintelligent AI could make conflicts worse or impose unwanted moral choices on everyone.