MECoBench tests when AI teamwork beats solo agents in embodied tasks

A team of researchers led by Qingyun Liu has unveiled MECoBench, a systematic benchmark designed to study how multimodal large language models (MLLMs) collaborate as embodied agents in visually grounded environments. While MLLMs have shown strong potential as single agents, their ability to work together in real-world settings has been underexplored. MECoBench addresses this gap by providing an evaluation platform that spans diverse real-world tasks, two distinct cooperation structures, and three collaboration modes. This allows for rigorous, reproducible testing of multi-agent collaboration in environments where agents must perceive, reason, and act together.

Through extensive experiments across various MLLMs, the researchers derived three key insights. First, collaboration generally improves embodied task completion, but the benefit hinges on balancing collaborative gains against the overhead of coordination complexity. Second, communication is a critical enabler of collaboration gains—without it, performance degrades—and the optimal collaboration mode varies with team size and the underlying model's capability. Third, collaboration actually enhances robustness under noisy priors and exploration conditions, suggesting that teams of agents can compensate for individual weaknesses. MECoBench thus offers the first systematic testbed for understanding the mechanisms and limits of multimodal embodied collaboration, with code and dataset publicly available.