AI Safety Experts Want a 'Kill Switch' Before AI Gets Too Smart

Most of us know AI as the thing behind ChatGPT. But the biggest AI companies are chasing something much bigger: "superintelligence" — AI that can outdo humans at everything, including hacking, lying to people, and running military operations. The safety group ControlAI just published a plan saying governments need to prepare for the harm this could cause, starting now.

The warning isn't coming only from outsiders. Two researchers at leading AI labs quit in recent weeks, saying the companies are racing toward something that could kill us all by the end of the decade. Elon Musk, who also builds AI, puts the odds of AI wiping out humanity at 10 to 20 percent. More than 800 scientists, former military leaders, and public figures have called for a ban on building superintelligence.

ControlAI's plan has three practical pieces. First, treat the most powerful AI systems like weapons and protect them from thieves. Second, if a company's AI leaks or escapes, hold that company criminally responsible — the same way we treat reckless handling of dangerous materials. Third, require a "kill switch": a reliable off button the government can hit during a serious AI incident. ControlAI has already helped draft kill-switch rules in the UK and backed a bipartisan kill-switch bill in the US.

The catch: the authors admit none of this actually fixes the problem. These are stopgaps — they soften the damage while superintelligence keeps being built. They also mean more government power over private companies, which could slow innovation or be misused. In their view, the only real fix is a worldwide ban, verified by international inspectors. Without that, they say, the risk of catastrophe remains.