Robots Can Now Learn a New Skill From One Demo

POIL is a point-based one-shot imitation learning framework with stable dynamical systems, from Sang Min Kim and five other authors. One-shot imitation avoids collecting extensive demonstrations, but successful one-shot manipulation requires both transferring a demonstrated trajectory to a novel object and executing it robustly under changing scene conditions, grasp configurations, and external disturbances. POIL tackles both through a shared representation: a set of 3D points on the object's functional part, used jointly for trajectory transfer and closed-loop execution. The one-shot transfer from the demonstrated trajectory is enabled with point correspondences, and POIL grounds that shared functional part with a multi-modal large language model, transferring the trajectory across viewpoint, pose, and object category changes. During execution, multi-view tracking observes the same points online, and Point-set BCSDM drives them in closed loop by projecting per-point velocities onto a single rigid-body twist computed from the tracked points alone — extending stable dynamical models from an SE(3) pose to a point set without needing a known 3D model or pose estimator. The authors show that at the goal the controller becomes a gradient flow on the classical SO(3) potential, so its terminal phase inherits that potential's almost-global convergence under a rigid-object assumption. Across simulation and real-robot experiments, POIL transfers a single demonstration across object category, grasp pose, and goal geometry, while recovering from external disturbances during execution.