AxisGuide: Grounding robot action coordinates in RGB images for robust manipulation
Even simple pickup tasks fail when object positions change – AxisGuide adds explicit coordinate cues.
Visuomotor manipulation policies trained via large-scale behavior cloning often fail under distribution shifts — even simple pickup tasks degrade when objects are at unseen locations. The core issue, argue researchers from several institutions, is insufficient action understanding: the robot can't interpret its base-frame coordinate system in image space. To address this, they introduce AxisGuide, a lightweight method using camera parameters and end-effector poses to render the robot's base-frame axes (+x, +y, +z) directly into each camera view. These explicit cue channels augment standard RGB observations, helping the model bridge semantic understanding and precise motor commands.
Extensive evaluations in both the LIBERO simulation and real-world environments demonstrate that AxisGuide yields substantial performance gains and improved generalization across unseen object positions and configurations. The method is accepted to RSS 2026, a top robotics conference. By providing explicit coordinate cues without heavy computational overhead, AxisGuide could become a standard plug-in for training more reliable generalist visuomotor policies, especially in production environments where distribution shifts are common.
- AxisGuide renders robot base-frame axes (+x, +y, +z) in camera views as explicit cue channels.
- Lightweight method uses only camera parameters and end-effector poses — no extra hardware or heavy computation.
- Tested on LIBERO simulation and real-world tasks, accepted to RSS 2026, showing robust generalization under object position shifts.
Why It Matters
Simple coordinate cues dramatically improve robot manipulation reliability, a practical fix for real-world deployment failures.