EgoSteer's VLA model teaches robot hands dexterity from 9.6K hours of video

EgoSteer addresses a critical bottleneck in dexterous robotics: the lack of large-scale, language-aligned demonstration data for steerable policies. The system integrates EgoSmith, a data pipeline that transforms in-the-wild egocentric videos into 9.6K hours of high-quality pre-training data with 9x higher throughput and better accuracy than previous state-of-the-art methods. This pipeline enables scalable data collection without expensive robot teleoperation, leveraging human video for language-guided manipulation priors.

On top of this data foundation, the team built EgoSteer, a world-model-enhanced vision-language-action (VLA) model trained using optimized infrastructure. After pre-training on human data, the model is grounded through robot post-training with DAgger refinement. Empirically, EgoSteer achieves robust execution of free-form instructions across 40+ diverse tasks, demonstrating failure recovery, dexterity, and generalization. The pre-trained model also few-shot adapts to complex long-horizon tasks like box folding on two robot embodiments with over 75% success. The entire system, data, and model are open-sourced.