GelSight Mini force mapping works across sensors without retraining

GelSight Mini tactile sensors are still largely handmade, causing unit-to-unit variations that traditionally force researchers to collect new data and retrain models for every sensor. In a new paper accepted at IEEE Sensors Letters, Julio Castaño Amoros and Pablo Gil propose a zero-shot transfer method that generalizes 3D force map estimation across different GelSight Mini units—even across sensor versions.

The approach uses a two-stage pipeline: first, a UniT-based model reconstructs the raw tactile image into a general tactile image to normalize sensor-specific artifacts. Second, a U-Net network estimates the 3D force map from that standardized representation. The authors report strong results: an SSIM of 0.9338 ± 0.0358 in the image reconstruction stage and a mean absolute force error (MAE_F) of 1.1294 ± 1.5934 N in force estimation. This means a model trained on one sensor can directly work on a new, unseen sensor without any fine-tuning, significantly reducing the calibration burden in robotics and teleoperation applications.