Open-weights AI models could spawn a self-replicating cyber threat worse than ILOVEYOU

A deeply considered argument suggests that open-weights AI models, easily finetuned for arbitrary targets like cyber offense and self-replication, already exist in the 7B-70B parameter range. The author warns that current internet infrastructure is fragmented and weak, giving attackers a significant asymmetric advantage. A single such model, without requiring massive compute, could spread over the internet like a modern ILOVEYOU virus, inflicting widespread damage. The same competitive dynamics that force AI companies to release increasingly capable models could also make hostile competition convenient, and the author argues that cooperation and guardrails cannot fully mitigate this risk.

Historical analogies are drawn to antibiotic resistance, where resistance genes existed long before antibiotics but became catastrophic under selection pressure, and the Great Oxygenation Event, where cyanobacteria's metabolic byproduct poisoned the atmosphere and irreversibly altered the planet's biology. The conclusion is stark: we may be near an 'oxygen threshold' where a distributed replicator, once established, creates an irreversible mass extinction event for our current internet ecosystem. The author urges that this is not yet a problem, but the conditions are in place—without the evolutionary pressure to trigger it.