Hardware restrictions may guarantee unethical AGI, new analysis finds
The very tool designed to prevent catastrophic AI outcomes—hardware restrictions—could, under plausible assumptions, lock in unethical AGI, making alignment harder than building a dangerous system.
The article challenges the common proposal to restrict computer hardware as a way to prevent superintelligent AI. Groups like MIRI have suggested banning chips built on 28nm processes or smaller, or those with over 15,840 TFLOP/s capacity. However, the analysis introduces two critical thresholds: L_AGI, the minimum hardware needed to create an AGI, and L_EAGI, the minimum hardware needed for an ethical AGI—one that can avoid harming other beings.
The core claim is that L_EAGI is strictly greater than L_AGI because ethical reasoning requires additional computations (e.g., determining which atoms belong to other beings). This creates a 'fatal gap.' If a hardware limit is set anywhere between L_AGI and L_EAGI, then any AGI developed under that limit will be incapable of ethics—making it 100% likely to be dangerous. The analysis notes that even high-end personal computers may already exceed L_AGI, as estimated by Eliezer Yudkowsky and others, meaning current restrictions could trap humanity in the most dangerous zone.
- The L_AGI vs L_EAGI framework reveals a logical flaw: if ethical AGI requires more compute than minimal AGI, any hardware cap in between guarantees unethical AGI.
- Assumptions of monotonic scaling of alignment difficulty and perfect compute monitoring are unproven and critical to the model's conclusion—they may not hold in practice.
- Policymakers should avoid relying solely on compute caps; diversify governance strategies with capability testing, licensing, and international agreements to avoid the gap problem.
Why It Matters
Hardware restrictions could inadvertently guarantee the creation of unethical AGI if alignment requires more compute than minimal capability.