Study Finds AI Agents Get Little Safety Testing Before You Use Them

AI agents (AI that can take actions for you, not just chat) are spreading fast. They can browse websites, run code, send emails, and remember things between tasks. That power is exactly what makes them risky: an agent that can log into your account can also be tricked into doing something you never asked for. A team of researchers looked at 157 open-source agent projects — open source means the code is public, so anyone can inspect and reuse it — each popular enough to have at least 100 GitHub stars, a rough measure of developer interest.

The team read the documentation, code, settings, and tests behind each project. What they found was uneven. Most projects check that the main features work and that obviously dangerous actions, like deleting files or spending money, are blocked. But the safety checks are applied inconsistently: one route into a capability might be guarded while another equal route is wide open. Tests almost never probe edge cases — weird inputs, deliberately hostile prompts, or failures that only appear after several steps of tool use, like an agent that gets slightly wrong information early and then acts on it confidently.

Most striking, the researchers say risks that developers clearly identified were rarely turned into actual end-to-end checks. In plain terms, teams wrote down the danger but didn't build the test that would catch it happening. This is like a restaurant posting an allergy warning but never checking whether the kitchen actually avoids the ingredient.

The fix the authors recommend is a shift from testing features to testing whole journeys: run the agent through complete, realistic workflows with untrusted input, outside tools, saved memory, and outside services, and confirm it stays inside the boundaries you set. Until that becomes normal, expect AI agents to keep surprising their users — sometimes helpfully, sometimes expensively.