FELT generates tactile signals from vision, boosting robot manipulation

A team of researchers led by Zinan Li has developed FELT (Feature-Extracted Latent Tactile), a learning-based framework that generates tactile signals from visual input alone. The system uses a large frozen visual encoder paired with a lightweight query decoder to predict per-finger pressure images in a single feed-forward pass. Unlike prior approaches, FELT respects the physical topology of dual-finger tactile sensors by decoding left and right panels through separate branches, capturing asymmetric contact patterns during tasks such as wiping, insertion, and in-hand rotation. This design allows the model to augment existing vision-only datasets with synthetic tactile data, either as generated images or as latent features.

In experiments on four contact-rich manipulation tasks, FELT's generated tactile signals consistently improved policy success rates compared to vision-only baselines. Notably, using latent tactile features required no real tactile sensor during policy training or deployment, lowering the barrier for collecting tactile data. The approach addresses a key bottleneck in visuo-tactile manipulation: tactile sensors are fragile, expensive, and non-standardized, making large-scale data collection difficult. By enabling robots to infer touch from vision, FELT paves the way for more robust manipulation in cluttered or partially occluded environments, with potential applications in manufacturing, healthcare, and household robotics.