TeleDex: Accessible Dexterous Teleoperation

A team of researchers has introduced TeleDex, an open-source system designed to make dexterous robot teleoperation radically more accessible. The core innovation is using a standard smartphone as the sole input device. The system streams low-latency 6-DoF (degrees of freedom) wrist poses and detailed 21-DoF hand state estimates directly from the phone's camera, eliminating the need for expensive external motion capture systems or specialized VR gloves. These human motions are then retargeted in real-time to control robotic arms and multi-fingered hands.

TeleDex offers two operational modes: a basic handheld phone mode and an optional 3D-printable hand-mounted interface for finer, finger-level control. This flexibility addresses a critical bottleneck in robotics: the difficulty of collecting high-quality, real-world demonstration data to train or fine-tune manipulation policies. By dramatically lowering the cost and technical setup required, TeleDex enables researchers and developers to quickly gather task-specific demos during actual robot deployment, directly supporting policy adaptation for new environments or tasks.

The system has been validated across both simulation and real-world manipulation tasks, proving its effectiveness as a unified and scalable teleoperation interface. All software, hardware designs, and demonstration videos are publicly available, encouraging widespread adoption and community development. This approach represents a significant step toward more practical and iterative robot learning, where policies can be continuously improved with human guidance without prohibitive infrastructure costs.