The People Building AI Say There's an 18% Chance It Ends Humanity

A nonprofit called AI Impacts asked 1,580 researchers who publish at top AI conferences what they really think about the future of their field. The survey has run every few years since 2016, and the latest answers came in December 2024. The results, just released, show that the people closest to the technology are deeply uncertain — and often alarmed.

The most striking number: on average, researchers gave an 18% chance that future AI could cause human extinction or permanently strip humans of control. That's roughly a 1-in-5 chance. More than half put the risk at 10% or higher, and one in three said at least 20%. In other words, the people building the technology aren't just worried about glitches or job losses — many are worried about the survival of humanity itself.

Another big finding: most researchers doubt that by 2029, users will be able to know the true reasons an AI system made a particular choice. That matters because AI is increasingly handed real decisions — who gets a loan, what medical care you receive, what news you see. It also highlights something most people don't realize: the people who build these systems don't program every rule. The AI learns on its own, and even its creators often can't explain what's going on inside.

The survey also found that researchers educated in Asia were more worried about catastrophe than those in the US or Europe — a surprising result given that global competition to build AI faster is often framed as a race with China. Overall, the picture is of a field that sees enormous promise but also real danger, with no agreement on what to do about it.