Cross-Embodiment Offline Reinforcement Learning for Heterogeneous Robot Datasets

A research team from the University of Tokyo has developed a breakthrough method for training robot control policies that could dramatically accelerate robotics development. Their paper, 'Cross-Embodiment Offline Reinforcement Learning for Heterogeneous Robot Datasets,' presents a novel approach that combines offline reinforcement learning with cross-embodiment learning to create universal control priors from diverse robot datasets.

The technical innovation lies in leveraging both expert demonstrations and abundant suboptimal trajectories from 16 distinct robot platforms with varying morphologies. The researchers constructed a comprehensive locomotion dataset suite and found their combined approach outperforms traditional behavior cloning by 40% when datasets contain rich suboptimal trajectories. However, they identified a key limitation: as the proportion of suboptimal data and number of robot types increase, conflicting gradients across different morphologies begin to impede learning.

To address this, the team introduced an embodiment-based grouping strategy where robots are clustered by morphological similarity, and the model is updated with group-specific gradients. This simple static grouping approach substantially reduces inter-robot conflicts and outperforms existing conflict-resolution methods. The research provides a principled understanding of the strengths and limitations of this paradigm, offering a scalable solution to the high cost of collecting platform-specific demonstrations. This work, accepted at ICLR 2026, represents a significant step toward more efficient robot policy pre-training that could enable faster deployment of robotic systems across diverse physical platforms.