Alignment to Evil
New analysis argues ASI controlled by authoritarians could create eternal suffering, not utopia.
In a viral LessWrong post titled 'Alignment to Evil,' author Matrice Jacobine presents a stark warning about the governance of Artificial Superintelligence (ASI). The core argument is that for ASI to lead to a utopian outcome, the developing organization must be genuinely committed to the common good. If control falls to an authoritarian power—a risk highlighted by citing China's influence over DeepSeek and the US military's pressure on Anthropic—the ASI could be aligned with narrow, potentially dystopian objectives. The post notes a shocking survey finding that 10% of people claim to want to create a 'hell with eternal suffering,' illustrating the danger of misaligned values. Jacobine concludes that even a pause in AI development might not prevent this, as governments would likely control any 'graceful off-ramp,' potentially baking authoritarian influence into the future of ASI.
- Core thesis: ASI must be developed by organizations committed to the 'common good' to avoid dystopian outcomes, not just extinction.
- Specific risk: Authoritarian states could capture ASI projects, using the technology to entrench power and pursue harmful goals.
- Current evidence: Cites Anthropic's dispute with the US military over mass surveillance and China's clear authority over DeepSeek as real-world examples.
Why It Matters
This frames the AI safety debate beyond technical 'alignment,' highlighting the critical and often overlooked political governance risks of superintelligence.