Tactus beats supervised CNN on touch recognition with 187 samples

A new open model called Tactus, created by Abdul Basit Tonmoy, tackles a long-standing problem in tactile robotics: recognizing objects from touch alone without expensive optical sensors. Resistive pressure arrays are the cheapest and most widely used tactile sensors, yet most representation learning has focused on optical gel-deformation sensors. Tactus flips that by answering natural-language text queries directly from pressure data. On the STAG benchmark with 27 objects and held-out recordings, it achieves 0.771 ± 0.062 top-1 accuracy (top-3 0.935), slightly beating the dataset's supervised closed-set CNN at 0.76—and it does this without any trained classification head.

The recipe is deliberately data-efficient: just 187 training recordings, combined with masked-autoencoder pretraining on 144k unlabeled frames from the same sensor, plus the sensor's own calibration affine transform. That calibration step alone recovered more accuracy than every architecture change combined. The model is also robust to phrasing—paraphrased or even bare-name queries perform within one point—and using two diverse frames recovers 89% of the accuracy of eight frames. The paper's honest failure analysis reveals that cross-sensor pretraining pooling gave no gain, and vision co-training actually degraded touch performance. A mis-normalized input pipeline once silently discarded 97% of the sensor's dynamic range while producing plausible results. Tactus's weights, code, and the memory layer it plugs into are released openly, letting robotics researchers reproduce and extend the results on low-cost hardware.