New Safety Test for AI Agents That Handle Your Money and Data

agents are programs that can do things for you—like transfer money, book a hotel, or answer emails—by using tools and changing data. They're becoming common in banking, shopping, and coding. But because they can take actions, they're a target for hackers. If an agent is tricked, it could leak your private info or let someone else control your account. So security testing is crucial.

Existing tests often focus on the AI itself, not the whole app. GreyZone is different: it tests real, open-source agent applications that have security boundaries (like who can access what). It checks if automated red-teaming services (tools that simulate attacks) can actually break those boundaries. The benchmark includes diverse apps, some with multiple agents working together or handling images and text. The goal is to see if attacks succeed and cause real violations, not just theoretical ones.

In two early tests, researchers used a tool called Promptfoo on two apps, Letta Code and Clawith. They looked at how attacks were delivered, how the agents behaved, and whether the attacks caused verified security breaches. This shows the benchmark works and can be used to improve agent security. The team released the benchmark publicly so others can research and build safer agents.

For you, this means that as AI agents become more integrated into daily life, efforts like GreyZone help ensure they're tested against real-world threats. It's like a crash test for AI helpers—making sure they don't crash when someone tries to hack them. This could lead to safer apps that protect your data and money.