95% Accurate Smart Glove Translates Sign Language Using Deep Learning
Flex sensors, IMU, and a camera combine to achieve 95% real-time sign recognition.
A team of five researchers from Vietnamese institutions has presented a deployable smart glove system for sign language recognition that overcomes real-world deployment challenges faced by earlier systems. The glove combines wearable sensors—flex sensors and an inertial measurement unit (IMU)—to capture detailed finger articulation and hand motion, while a camera adds facial cues for richer context. Sensor data is transmitted via an ESP32-C6 microcontroller and processed by a long short-term memory (LSTM) network, which models temporal gesture dynamics with an overall recognition accuracy of approximately 95%.
The trained model is further converted to TensorFlow Lite for real-time inference on edge devices, making the system practical for everyday use. The paper, submitted to the IFToMM International Symposium on Robotics and Mechatronics, highlights the feasibility of low-cost, wearable AI for bridging communication gaps. This system represents a significant step toward accessible assistive technology, potentially enabling deaf individuals to communicate more freely in diverse environments without relying on bulky hardware or cloud processing.
- Integrates flex sensors, IMU, and camera for multimodal gesture capture including hand motion and facial cues.
- Uses LSTM deep learning network on ESP32-C6 microcontroller to achieve 95% recognition accuracy.
- Model converted to TensorFlow Lite for real-time inference, enabling practical on-device translation.
Why It Matters
Brings wearable AI closer to bridging communication gaps for deaf individuals with real-time sign language translation.