Yingying Wang's Acoustic Robots Achieve 96% Success in Contactless Manipulation

Yingying Wang and her team have developed a groundbreaking decentralized framework for coordinating acoustic robots, leveraging natural language processing to facilitate contactless object manipulation. This innovative system employs Whisper-based speech recognition to convert spoken instructions into actionable multi-robot task plans. By using LLMs for semantic parsing and structured JSON representations for task assignments, the framework enables robots to execute tasks across various scenarios, achieving impressive success rates of 96% for sequential tasks, 86% for parallel execution, and 70% for synchronized collaborative transport.

The implementation was demonstrated using two TurtleBot3-based acoustic robots equipped with ultrasonic phased arrays. The framework allows non-expert users to interact seamlessly with multi-robot systems by issuing high-level commands, which are translated into detailed execution plans. This research highlights the potential of integrating large language models with distributed robotic systems, paving the way for advanced automation in fields such as healthcare and laboratory environments. By simplifying user interaction and enhancing efficiency, this approach could revolutionize how robots are utilized in precision transport and other critical applications.