PLUME: New world model helps robots adapt to uncertain physics in dexterous tasks

Precise multi-finger manipulation, such as turning a screwdriver or lifting a bucket, is highly sensitive to physical parameters like object shape, pose, and friction coefficients. While simulation allows massive data collection with known parameters, real-world robots face uncertainty—standard domain randomization often fails for tasks requiring parameter-specific strategies. To solve this, a team of researchers from the University of Michigan (Abhinav Kumar, Soshi Iba, Rana Soltani Zarrin, Dmitry Berenson) proposes PLUME (Probabilistic Latent Unified World Modeling and Parameter Estimation). PLUME learns a latent space that jointly represents multiple physical parameters and rewards, enabling a world model to evolve a belief over parameter values and dynamics simultaneously. This allows the robot to infer true parameters online during deployment and adapt its policy without retraining.

PLUME was evaluated on four simulated tasks—screwdriver turning, valve turning, bucket lifting, and disk flicking—as well as a real hardware screwdriver turning experiment. It achieved zero-shot transfer from simulation to reality, outperforming state-of-the-art offline reinforcement learning and behavior cloning baselines. The method is particularly effective for tasks requiring precise, parameter-conditioned strategies. By treating parameter estimation as part of the world model, PLUME bridges the sim-to-real gap for dexterous manipulation, promising more robust and adaptable robotic hands in manufacturing, healthcare, and domestic settings.