DESBench benchmark reveals coordination trade-offs for multi-agent systems

A new research paper, 'When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling,' introduces DESBench, a Distributed Event-driven Scheduling Benchmark designed to evaluate how different coordination paradigms perform in complex, shared environments. Unlike existing multi-agent benchmarks that focus on weakly coupled tasks, DESBench captures the realities of industrial scheduling: multi-timescale decision making, partial observability, and dynamically coupled constraints. The benchmark tests four representative coordination paradigms — centralized, hierarchical, heterarchical, and holonic — each with distinct mechanisms for information flow, decision authority, and conflict resolution. Controlled evaluations on DESBench reveal clear trade-offs that go beyond simple outcome metrics. Centralized coordination is robust and communication-efficient but fails to scale with problem difficulty. Hierarchical coordination improves efficiency through decomposition but suffers from cross-level misalignment, where decisions at different hierarchy levels conflict. Heterarchical (peer-to-peer) coordination is flexible but incurs heavy communication overhead. Holonic coordination (where agents form nested groups) satisfies constraints well but loses global robustness. These results underscore that coordination design fundamentally shapes agent system behavior in complex environments, and that existing outcome-only metrics miss critical structural trade-offs.

The findings have direct implications for real-world multi-agent system (MAS) deployments, especially in manufacturing, logistics, and supply chain management where industrial scheduling is critical. The paper argues for more adaptive, principled, and dynamic coordination mechanisms that can switch between paradigms as conditions change. DESBench provides a standardized testbed for future research, enabling researchers to quantitatively compare coordination strategies across multiple dimensions: effectiveness, constraint alignment, coordination efficiency, and robustness. As AI agents increasingly collaborate in shared physical and digital environments, understanding when hierarchy helps — or hurts — becomes essential for building reliable, scalable multi-agent systems.