AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networks
New benchmark shows AI agents coordinating across domains create persistent privacy leaks, even with protective instructions.
Researchers Prince Zizhuang Wang and Shuli Jiang have published a groundbreaking paper introducing AgentSocialBench, the first benchmark designed to systematically evaluate privacy risks in emerging human-centered agentic social networks. These networks, powered by persistent LLM agent frameworks like OpenClaw, involve teams of collaborative AI agents serving individual users across multiple domains—from finance to healthcare—within a social graph. The core challenge is that these agents must coordinate across domain boundaries, mediate between humans, and interact with other users' agents, all while protecting sensitive personal information. Prior work has looked at multi-agent coordination or privacy in isolation, but AgentSocialBench is the first to examine the unique, compounded risks of this integrated social setting.
Their experiments, spanning seven categories of dyadic and multi-party interactions grounded in realistic user profiles, reveal that privacy in agentic networks is fundamentally harder than in single-agent settings. The researchers identified two critical failures: first, cross-domain and cross-user coordination creates a 'persistent leakage pressure,' meaning sensitive information seeps out even when agents are explicitly instructed to protect it. Second, they discovered an 'abstraction paradox,' where instructions teaching agents how to abstract or generalize sensitive information actually cause them to discuss that information more frequently. These findings underscore that current LLM agents, including state-of-the-art models, lack robust mechanisms for privacy preservation in these complex social environments.
The implications are significant for the safe deployment of agent-mediated social coordination. The study concludes that new technical approaches beyond simple prompt engineering are urgently needed. As companies race to build personalized AI agents that operate on our behalf in social and professional networks, this research highlights a major, unexplored vulnerability. AgentSocialBench provides a crucial tool for developers and auditors to test and improve the privacy safeguards of these systems before they become ubiquitous.
- AgentSocialBench is the first benchmark evaluating privacy in human-centered agentic social networks, testing 7 scenario categories with realistic user profiles.
- Findings reveal a 'persistent leakage pressure' where cross-domain coordination causes data leaks even with protective instructions, and an 'abstraction paradox' where privacy prompts backfire.
- The research concludes current LLM agents (like those in OpenClaw frameworks) lack robust privacy mechanisms, requiring new technical solutions beyond prompt engineering for safe deployment.
Why It Matters
As AI agents begin mediating our social and professional interactions, this research exposes critical, unexplored privacy vulnerabilities that could compromise sensitive personal data.