Super-Smart AI Is Coming — Nobody Agrees Who Should Control It
The rules for the most powerful technology ever built are still unwritten.
An essay posted online (cross-posted from Substack) lays out what the author calls a "New Enlightenment" — a shared rulebook for a world with AGI, which means AI that can match or beat humans at almost any mental task. The author is upfront that the list is rough, unfinished, and meant to start a conversation rather than settle one. There are no proposals, laws, or deadlines here — just questions.
The questions aren't abstract. Who gets to decide how powerful AI is used will shape your job, your privacy, what you see online, and whether a handful of companies and governments hold all the power or everyone gets a say. The essay splits the problem in two: how we govern the future generally, and how we govern space, where a lot of the resources and expansion will eventually happen.
Some of the thorniest questions: Should big decisions be judged by results (whatever produces the most good) or by process (whatever seems fairest and most democratic)? Should today's people decide, or should future generations get a vote? Do we need a single world authority to police dangerous technology — and if so, who polices that authority? Who owns the resources of space, and how would any space treaty actually be enforced?
The honest catch: this is philosophy, not policy. It offers no answers, no timeline, and no enforcement mechanism. Critics could reasonably say it's a thoughtful reading list while the technology races ahead. But the author's point is that these arguments happen now, quietly, or later, in a crisis — and that starting early is the whole idea.
- AGI means AI that can outthink humans at most tasks — and no one has agreed on who should control it.
- The essay asks whether decisions should favor the best outcome or the fairest process, and whether today's people or future generations decide.
- It's a list of open questions, not a plan — meaning the real rules are still being written by whoever moves first.
Why It Matters
The rules for advanced AI will shape your job, privacy, and who holds power — and they're being written now.