VLCP's closed-loop code replanning boosts robot success 10x

Turning a frontier vision-language model (VLM) into a robot policy usually means fine-tuning it to emit an action representation it never saw in pretraining—discarding much of the reasoning that made the model valuable. VLCP, from Dhia Naouali and four collaborators, flips this approach: it keeps the VLM frozen and has it write policy as a short Python control function, with no demonstrations and no fine-tuning. But writing that code once is open-loop; existing closed-loop methods react at the wrong level by retrying a fixed policy or picking a different subtask, never rewriting the code that failed.

VLCP closes the loop at the control code level within a single episode. Every K steps, the VLM re-observes the scene from multi-view RGB, proprioceptive state, and a state delta, then rewrites the control function from what it just saw, catching failures before they compound. On a 57-task MuJoCo/RoboVerse sweep, the training-free policy reaches 35.1% pooled success versus 3.5% for the identical system queried once per episode—a tenfold gap with non-overlapping confidence intervals across every scene family. The gain traces to a 27.3% within-episode recovery rate on failed grasps, converting misses an open-loop controller would carry to episode end. The loop stays cheap: 84% of input tokens hit cache, episodes need only about 10 compact queries, and control blocks persist to a cross-episode skill library for future prompts.