RACAS: Controlling Diverse Robots With a Single Agentic System
A single agentic AI system successfully commanded a ground robot, underwater vehicle, and multi-jointed limb.
A research team including Dylan R. Ashley and AI pioneer Jürgen Schmidhuber has published RACAS (Robot-Agnostic Control via Agentic Systems), a breakthrough in universal robot control. The system uses a cooperative agentic architecture where three specialized modules—Monitors, a Controller, and a Memory Curator—communicate exclusively through natural language to provide closed-loop control. Unlike existing approaches that require retraining for each new robot or only work on similar platforms, RACAS needs just three inputs: a natural language description of the robot, a definition of available actions, and a task specification. Crucially, it requires no modifications to source code, model weights, or reward functions when switching between radically different robotic embodiments.
The team validated RACAS on three diverse platforms: a standard wheeled ground robot, a recently published novel multi-jointed robotic limb with complex articulation, and an underwater vehicle operating in a different physical medium. Despite these fundamental differences in embodiment and environment, RACAS consistently solved all assigned tasks across all three platforms. This demonstrates that agentic AI systems using natural language as their communication medium can achieve remarkable generalization in robotics. The 7-page paper, submitted to arXiv, shows how this approach could substantially reduce the expertise barrier and development time required for prototyping robotic solutions, moving researchers closer to truly general-purpose robot control systems.
- Uses three LLM/VLM modules communicating through natural language for closed-loop control
- Requires only robot description, action definitions, and task specs—no code or weight changes
- Successfully controlled three radically different robots: ground, underwater, and multi-jointed limb
Why It Matters
Could dramatically reduce robotics prototyping time and expertise barriers by enabling single AI systems to control diverse robots without retraining.