Advancing Multi-Robot Networks via MLLM-Driven Sensing, Communication, and Computation: A Comprehensive Survey

A team of 13 researchers, led by Hyun Jong Yang, has published a forward-looking survey titled 'Advancing Multi-Robot Networks via MLLM-Driven Sensing, Communication, and Computation.' The paper introduces the concept of 'R2X' (Resource-to-Anything), framing multi-robot coordination as an intent-to-resource orchestration problem. The core idea is that a system-level orchestrator, powered by a Multimodal Large Language Model (MLLM), can interpret a high-level natural language command (e.g., 'inspect the warehouse for spills') and use that understanding to dynamically optimize which sensors to activate, how much bandwidth to allocate, and where to place computation. This prevents networks from being overwhelmed by raw sensor data streams.

The survey reviews state-of-the-art techniques and presents four end-to-end demonstrations of the R2X concept. These include a digital-twin warehouse navigation system with predictive link control, a 'FollowMe' robot with a semantic-sensing switch, and a real-hardware system for open-vocabulary trash sorting using edge-assisted MLLMs. The authors emphasize that splitting reasoning between on-device models and more powerful edge or cloud servers, guided by the MLLM's understanding of intent, leads to superior system-level performance. They measure success through key metrics like payload size, latency, and task success rate, showing that this orchestrated approach outperforms purely on-device AI baselines. The work provides a roadmap for building scalable, efficient teams of robots that can handle complex, real-world missions.