LessWrong's MichaelDickens argues frontier AI company should voluntarily shut down to force regulation

In a provocative LessWrong post, MichaelDickens calls for a frontier AI company—any one, he emphasizes—to shut down and announce that powerful AI could end the human race and that nobody knows how to make it safe. He argues that current safety-conscious companies are trapped in a race dynamic: they say they'd slow down but fear competitors will rush ahead. A unilateral shutdown, he claims, would send a signal so strong that governments must take extinction risk seriously, potentially spurring coordination and sane regulation. He dismisses the common refrain that stopping doesn't matter because others will build anyway, noting that a multi-hundred-billion-dollar company publicly admitting its product is horribly dangerous would raise eyebrows and change narratives.

Dickens addresses counterarguments: shutting down stops safety research, but he believes current safety efforts aren't reducing takeover risk enough to justify continued frontier development. He suggests an alternative: reallocate 100% of budget to safety research and global coordination until funds run out. He also tackles the legality for different corporate structures—Anthropic as a public benefit corporation likely could shut down without investor lawsuits, while OpenAI's for-profit/nonprofit hybrid is murkier. He concludes that 'we have to risk killing everyone because investors might sue' is indefensible. The post challenges the community to specify conditions—if not now, when—and insists that the status quo of racing ahead with a dash of safety research is even less likely to succeed than a dramatic shutdown.