Zero Shot Deformation Reconstruction for Soft Robots Using a Flexible Sensor Array and Cage Based 3D Gaussian Modeling

A research team has introduced a novel AI framework that allows soft robots to reconstruct their own 3D shape in real-time using only tactile sensing, eliminating the need for cameras. The system, detailed in the arXiv paper 'Zero Shot Deformation Reconstruction for Soft Robots Using a Flexible Sensor Array and Cage Based 3D Gaussian Modeling,' works by integrating a flexible piezoresistive sensor array with a geometry-aware, cage-based 3D Gaussian deformation model. Local touch measurements are mapped to control signals for a virtual 'cage' surrounding the object, which then deforms a cloud of 3D Gaussian primitives to render the robot's current shape.

Crucially, the method is 'zero-shot,' meaning it can generalize to previously unseen soft robots without collecting new training data or fine-tuning the model. It only requires a static 3D model (like an STL file) of the robot in its undeformed state. A graph attention network processes the tactile input to predict cage displacements, enforcing spatial smoothness. In tests, the system successfully reconstructed bending and twisting motions, achieving a 0.67 Intersection over Union (IoU) score, a 0.65 Structural Similarity Index (SSIM), and an average error of 3.48 mm in Chamfer distance. It also renders photorealistic RGB outputs in real time, providing a complete visual model derived purely from touch.