BIFROST enables zero-shot robot transfer from simulation to reality
New method learns invariant features to let robots skip retraining when moving sim-to-real.
Sim-to-real transfer remains a critical bottleneck in deploying robot policies learned in simulation. Existing approaches often stack separate modules to address visual and dynamics gaps, but the underlying assumption is that tasks share deep structure regardless of domain. BIFROST (Bridging Invariant Feature Representation for Observation-space Sim2Real Transfer) directly exploits that commonality. It trains a history encoder on paired cross-domain data using a cross-domain bisimulation objective: observation-action chains that yield equivalent long-term behavior are embedded into the same latent state, no matter the rendering or physics differences. Policies trained on those latent states in simulation can then be deployed in reality with zero additional training.
Empirically, BIFROST achieves effective transfer where strong baselines fail. In sim-to-sim visual navigation, it handles lighting and texture shifts. On a real-world contact-rich peg insertion task and a visual servoing task (both with both visual and dynamics domain gaps), BIFROST succeeds zero-shot while domain adaptation and co-training methods struggle or outright fail. The work shows that bisimulation can bridge the sim-to-real gap from raw observations, opening a path to more practical robot learning that requires little to no real-world data collection.
- Uses a cross-domain bisimulation objective to embed equivalent behavior sequences into shared latent states across simulation and reality.
- Achieves zero-shot transfer on both visual navigation and contact-rich manipulation tasks, handling combined visual and dynamics gaps.
- Outperforms domain adaptation and co-training baselines, which fail under multiple domain mismatches.
Why It Matters
This could slash the real-world data needed for robot training, accelerating deployment in factories and homes.