26 Top AI Researchers Warn of Automated Hacking, Fake Videos, and Killer Robots

In a 2018 report, 26 top AI researchers – including Shahar Avin from Cambridge’s Center for the Study of Existential Risk – warned that as AI systems begin to exceed human-level performance, the risks of misuse grow sharply. One imminent threat is automated spear-phishing: computers can now model a target’s writing style and preferences to craft perfectly tailored deceptive messages, eliminating the need for human effort. Another is the rise of hyper-realistic deepfakes. Developers at the University of Washington demonstrated an AI algorithm that could generate a video of Barack Obama saying anything the programmer wanted, and researchers caution that such synthetic media may soon become pixel-to-pixel indistinguishable from real footage, demanding new verification tools like digital signatures.

The report also highlights vulnerabilities in autonomous vehicles. While self-driving cars (e.g., Ford’s Argo) can navigate better than average human drivers, simple modifications like stickers on a stop sign can trick the AI into interpreting it as a “go” signal, creating a security flaw that malicious actors could exploit at scale. Beyond cars, drones and robotic systems could be repurposed into weapons – crashing fleets of autonomous vehicles, turning commercial drones into face-targeting missiles, or ransoming critical infrastructure. Warfare is perhaps the most disturbing application: the race for battlefield advantage will push militaries to deploy thousands of sensors and robots, generating terabytes of data that no human can aggregate. This could lead to decision-recommendation systems that remove meaningful human control over lethal force. The researchers also note the danger of AI-powered mass surveillance by oppressive regimes. They stress the technology’s many positive uses but insist that engineers must proactively address these dual-use risks – and that AI itself will likely provide many solutions.