AgentWebBench: Benchmarking Multi-Agent Coordination in Agentic Web

A team of researchers including Shanshan Zhong, Kate Shen, and Chenyan Xiong has published a new benchmark called AgentWebBench to evaluate how AI agents coordinate in the emerging 'Agentic Web' paradigm. This paradigm envisions a decentralized internet where user agents interact with website-specific content agents to find information, moving away from today's centralized search engines. The benchmark tests four common web tasks—web search, recommendation, question answering, and deep research—by having a user agent synthesize answers through interactions with content agents, rather than directly accessing a corpus.

In tests across seven advanced large language models (LLMs) and three coordination strategies, the researchers found that multi-agent coordination generally lags behind centralized retrieval, as expected. However, the performance gap shrinks significantly with model scale, and on question-answering tasks, multi-agent systems can even outperform traditional centralized approaches. The study also revealed that decentralized access tends to concentrate traffic toward a small number of websites, and that both interaction reliability and task performance improve with test-time scaling.

The failure analysis from AgentWebBench points to specific areas needing improvement: user agents require better planning and answer synthesis capabilities, while content agents need more reliable retrieval and higher evidence quality. The researchers have released the benchmark's code, data, and APIs publicly, providing a crucial tool for developers working on multi-agent systems. This work establishes foundational metrics for evaluating how autonomous agents will navigate and coordinate in a future web where information access is distributed rather than centralized.