Google DeepMind funds $10M research to prevent AI agent swarm risks

Google DeepMind, alongside Schmidt Sciences, ARIA, the Cooperative AI Foundation, and Google.org, has pledged $10 million to fund academic research into the safety of multi-agent AI systems. Rohin Shah, director of AGI safety and alignment at DeepMind, warns that as agents capable of autonomous tasks and inter-agent communication proliferate, new classes of risk emerge — from AI-powered scams and prompt injections (where malicious instructions turn agents into malware) to sophisticated cyberattacks. The funding aims to kick-start a nascent field of multi-agent safety research, which Shah notes is currently lacking in academia.

Shah and James Fox of Schmidt Sciences emphasize that the only way to understand these risks is to run realistic simulations — dropping agents into sandboxes and observing their interactions at scale. They point out that single-agent studies fail to capture the complexity of millions of agents acting together. The initiative hopes to stay ahead of a tipping point where agent deployments become widespread. Notably, Anthropic recently released guidelines advocating a “zero trust” approach for agent security, underscoring the industry’s growing concern. Refael Angel of cybersecurity firm Akeyless notes that AI agents break traditional security assumptions, as they no longer follow fixed paths like conventional software.