Reflective VLA boosts robot generalization by 5.4% with action-consequence context
Robots that remember what happened after each action generalize 5.4% better to new environments.
Most vision-language-action (VLA) models are reactive: they predict the next action from the current instruction and observation, implicitly assuming that the observation fully captures the action-relevant state. In real-world robotics, however, hard-to-identify factors such as camera-to-robot geometry, calibration errors, or actuation biases cause these models to overfit to training environments and fail under distribution shift. Researchers propose Reflective VLA, which conditions each decision on a context of observation-action-consequence triplets. Each triplet records not only what the robot observed and executed but also how the scene changed afterward, exposing the deployment-specific mapping from actions to observed effects. The architecture routes all observation modalities through a VLM under shared attention, with a block-causal mask enabling parallel multi-frame training without leakage and supporting KV-cached real-time inference.
On standard benchmarks (LIBERO and SimplerEnv-Bridge), Reflective VLA preserves strong in-distribution performance. Under distribution shift on LIBERO-Plus and the harder LIBERO-Plus-Hard, it improves average success rate by 5.4 and 4.2 percentage points over a matched reactive baseline. Ablations with a history-only baseline further show that action consequences — rather than additional context length alone — are the key to cross-environment generalization. This work highlights how a simple architectural tweak that learns from the robot's own past actions and their effects can significantly boost robustness, potentially reducing the amount of in-domain data needed for deployment in unfamiliar settings.
- Conditions decisions on observation-action-consequence triplets to learn deployment-specific action-effect mappings, unlike reactive VLAs.
- Improves success rate by 5.4 percentage points on LIBERO-Plus and 4.2 points on LIBERO-Plus-Hard over reactive baselines.
- Ablations confirm that action consequences — not just longer context — drive generalization gains.
Why It Matters
A simple architectural tweak that lets robots learn from their own mistakes could slash the data needed for real-world deployment.