New Trick Makes Robots See Faster Without Losing Accuracy
This could make robots cheaper and quicker, bringing them into more homes and jobs.
Robots using vision-language-action models can perform versatile manipulation tasks from language instructions, but inference cost limits how fast they respond in closed-loop control. A proposed training-free token pruning method moves the pruning decision ahead of the vision encoder by grounding it in a geometric prior from the robot’s own action.
The action chunk predicted at the previous control step is available before the current observation is encoded and specifies end-effector and camera motion between observations. In the fixed third-person view, the method retains patches along the projected end-effector trajectory plus semantic anchors; in the wrist view, it prunes patches that remain unchanged after compensating for action-induced camera motion. Experiments on LIBERO, VLABench, and four real-robot tasks show up to 1.61× speedup and 59.5% FLOPs reduction with negligible degradation in success rates.
- ActPrune speeds up robot vision by skipping unnecessary parts of the image, making robots respond faster.
- It uses the robot's planned movement to predict what to focus on, reducing computation by up to 60%.
- In tests, robots were up to 61% faster with almost no loss in task success, meaning quicker and cheaper robots.
Why It Matters
Faster, more efficient robots could soon assist in homes, hospitals, and factories, saving time and money.