Rapid Adaptation of Particle Dynamics for Generalized Deformable Object Mobile Manipulation

A team from Stanford University led by Bohan Wu, Roberto Martín-Martín, and Li Fei-Fei has introduced RAPiD (Rapid Adaptation of Particle Dynamics), a novel AI method that enables robots to manipulate soft, deformable objects whose material properties are initially unknown. The research addresses a critical gap in robotics, where traditional methods excel with rigid objects but struggle with materials like cloth, dough, or cables that change shape during interaction. The key innovation extends the Rapid Motor Adaptation (RMA) framework—previously successful in legged locomotion—to the complex domain of deformable objects by using ground-truth particle positions from simulation to capture shape changes.

RAPiD operates in two distinct phases. First, it trains a visuomotor policy in simulation, conditioning robot actions on a learned embedding of the object's dynamics, which includes privileged information like mass and the precise positions of simulated particles. Second, it learns to infer this same dynamics embedding using only non-privileged, real-world sensor data—specifically robot visual observations and past actions. This two-stage approach allows the policy trained in simulation to transfer effectively to physical robots. Tested on a mobile manipulator with 22 degrees of freedom, the system achieved success rates exceeding 80% on real-world, vision-based manipulation tasks, demonstrating robustness across different object dynamics, categories, and specific instances.