AI accelerates bioweapon risks before saving humanity, argues new analysis
AI progress may bring extinction-level pathogens within a decade
A new analysis by MichaelDickens on LessWrong challenges the argument that building aligned superintelligence quickly is necessary to protect humanity from other extinction risks like synthetic biology. The author contends that near-term AI progress will actually increase biorisk by accelerating scientific knowledge across the board. Even if AI systems refuse to directly aid in bioweapon development, the general advancement of biology brings the capability to engineer highly transmissible, lethal pathogens from zero to nonzero far sooner than would otherwise happen.
The analysis points out two minor issues with relying on AI refusal: current models are easily jailbroken, and at least one competitor will likely fail to lock down their model. The fatal issue is that AI doesn't have to participate in bioweapons creation—it merely accelerates humanity's collective understanding. This compresses a hundred years of progress into a decade, meaning an evil scientist could plausibly create an extinction-level pathogen by 2036. The window to identify risks and implement policy shrinks dramatically, making the near-term biological threat higher than in a world without AI acceleration.
- AI accelerates general scientific knowledge, moving bioweapon capability from zero to nonzero within a decade
- Current AI refusal mechanisms are easily bypassed, and competitors may not implement them at all
- The high-risk period for extinction-level pathogens starts sooner, leaving less time for defensive measures and policy responses
Why It Matters
AI progress may increase near-term existential risks before delivering long-term safeguards, demanding urgent biosecurity planning.