VBT-MPC framework lets robots feel and follow contours with vision-based touch
New tactile MPC eliminates separate pose estimation for precision contour tracking.
Researchers Velasco-Sanchez et al. propose VBT-MPC, a framework that enables robots to precisely follow object contours using a vision-based tactile sensor. The key innovation is operating the model predictive controller directly in contour features space, bypassing the need for separate pose-estimation modules or complicated force-control architectures. The sensor is mounted in an eye-in-hand configuration, letting the robot “feel” the surface while visually guiding the motion. This simplifies the control pipeline and improves accuracy on complex shapes.
The team compared VBT-MPC against visual-servoing strategies adapted to tactile features, testing on objects with varying geometries and materials (e.g., curves, edges, soft surfaces) in both simulation and real-world settings. The results showed reliable contour tracking even when the robot lost visual contact or encountered unexpected stiffness. The work has been accepted for publication in IEEE Robotics and Automation Letters and is supported by the REMAIN Project (Interreg-VI Sudoe). This approach could advance robotic surface inspection, polishing, and assembly tasks that require gentle, precise contact.
- VBT-MPC operates in contour feature space, eliminating separate pose-estimation modules.
- Uses a vision-based tactile sensor (VBTS) in an eye-in-hand configuration for direct feedback.
- Validated on diverse object geometries and materials in both simulation and real-world experiments.
Why It Matters
Simplifies robotic precision tasks like surface inspection and polishing by combining touch and vision in a single control loop.