FlowControl steers VLA robots with keyboard inputs, no retraining needed

A new paper from researchers Jonathan C. Kao, Jason Chan, and Andy Wang introduces Flow Control, a method for steering vision-language-action (VLA) models in real-time using generic inputs such as a keyboard. Unlike previous approaches that require costly retraining or fine-tuning, Flow Control works out-of-the-box, transforming crude user commands into action samples drawn from the VLA's expert action distribution. This ensures generated actions maintain high quality (conforming to the expert distribution) and high fidelity (reflecting user intent). The technique is demonstrated on robotic tasks, showing it accurately and responsively steers robot behavior even when user inputs are suboptimal.

Flow Control yields several desirable properties: it significantly improves task success rates and reduces completion times compared to autonomous operation alone. Additionally, fine-tuning a VLA on trajectories collected via Flow Control further enhances the model's autonomous policy, creating a virtuous cycle of human-in-the-loop improvement. The method is simple, intuitive, and does not require specialized hardware—just a keyboard or any low-dimensional input source. This opens the door for broader human-robot collaboration by allowing non-experts to guide sophisticated AI systems in real-world environments. The paper is available on arXiv under reference 2606.10180.