IWR algorithm boosts robot manipulation by 19.8% in simulation

Contrastive Reinforcement Learning (CRL) has excelled in locomotion and simple control but struggles with manipulation tasks that involve contact or grasping—what the researchers call “object-centric interaction.” The paper, led by Tongle Shen and colleagues, formalizes manipulation dynamics as a piecewise-smooth Markov process, arguing that interaction-induced mode changes create nonlinear reachability structures that standard CRL energy functions cannot represent well. Their solution, Interaction-weighted Resampling (IWR), performs targeted resampling around phases before, during, and after interactions, helping the learned representation capture multi-modal and piecewise nonlinear reachability boundaries.

Across several interaction-centric environments—including 2D dynamic control, robotic manipulation, and robot air hockey—IWR consistently outperformed prior CRL methods, achieving a 19.8% average improvement in simulation. The team then deployed IWR-trained policies in a sim-to-real pipeline for a real-world robot air hockey agent. This agent became the first goal-conditioned robot air hockey system capable of hitting goals, raising success rates from 25% to 60%. The work demonstrates that interaction-aware representation learning can bridge the sim-to-real gap for complex manipulation tasks, opening the door to more sample-efficient and physically capable robots.