COBALT lets anyone train robots with just a smartphone from anywhere

COBALT (Cloud-Based Teleoperation for Robot Learning) is a new platform from researchers at Georgia Tech and NVIDIA that dramatically lowers the barrier to collecting robot training data. Traditional robot teleoperation requires expensive, specialized hardware like haptic gloves or proprietary joysticks, limiting who can contribute. COBALT flips that by letting operators use their everyday smartphones (dual phones, one phone, VR headsets, 3D mice, or keyboards) to control robots in real time over the cloud. The system uses vectorized environments and a load-balanced infrastructure to support 8 concurrent teleoperators per single GPU with end-to-end latency under 100ms and a 20Hz control loop. Scaling across multiple GPUs, it can handle 256 simulated clients simultaneously.

To prove the concept, the team ran a 5-day global crowdsourcing campaign that collected 7,500+ demonstrations (over 50 hours) from nine countries using only smartphones. A comprehensive user study showed phone-based teleoperation performed comparably to or better than specialized hardware, with faster and more ergonomic data collection. The platform automatically filters low-quality demos using real-time metrics, and a structured training curriculum further boosted data quality. When used to train state-of-the-art imitation learning algorithms, the resulting models performed strongly, validating that crowdsourced, phone-driven teleoperation can scale robot learning to global participation.