Three Worlds Collide assumes calibration is solved
A viral AI safety story everyone cites is wrong about one crucial thing...
Deep Dive
Eliezer Yudkowsky's famous 2009 sci-fi story 'Three Worlds Collide' is a cornerstone of AI alignment debates, illustrating a tragic conflict between species with irreconcilable values. A new analysis argues the story's core assumption—that the alien species truly understand each other's moral reasoning—is a critical flaw. It highlights the difference between 'shared knowledge' and the 'common knowledge' needed for real coordination, suggesting we may misjudge superintelligent AI's intentions entirely.
Why It Matters
If we misunderstand AI's values, our safety strategies could be catastrophically misdirected from the start.