MIT's ultrasound wristband lets you wirelessly control robotic hands

MIT researchers, led by mechanical engineering professor Xuanhe Zhao, have created a wristband that uses a miniaturized ultrasound transducer paired with a hydrogel sticker to capture real-time images of the wearer's wrist muscles, tendons, and ligaments. An AI algorithm trained on human-labeled ultrasound data then maps these images to the corresponding positions of the five fingers and palm, enabling precise hand-tracking without bulky gloves or cameras. The system wirelessly transmits the data to control a robotic hand, letting wearers perform actions like playing a piano tune or shooting a mini basketball through a desktop hoop. The same wristband also allows manipulation of virtual objects on a screen, such as pinching to zoom.

Currently the hardware is about the size of a cell phone, but the researchers aim to miniaturize it further for practical wearability. They also plan to expand their training dataset to include more diverse hand sizes, finger shapes, and gestures, improving the AI's accuracy and generalization. Potential applications include training humanoid robots for delicate tasks like surgery, enabling intuitive control in virtual reality and video games, and providing a new interface for designers and engineers to interact with digital objects. The team believes this ultrasound-based approach is the most advanced method for tracking dexterous hand motion, offering a non-invasive, high-fidelity alternative to existing solutions.