ShapeGen: Robotic Data Generation for Category-Level Manipulation
A new method creates shape-varied robotic manipulation data without simulators, boosting real-world performance.
A team of researchers from Tsinghua University and other institutions has introduced ShapeGen, a groundbreaking method for generating the diverse training data robots need to handle the real world. The core challenge in robotics is category-level manipulation: teaching a robot to interact with any object in a category (e.g., any mug), not just the specific ones it was trained on. Manually collecting data on thousands of shape variations is prohibitively expensive and slow. ShapeGen solves this by automating data generation in a simulator-free, 3D-centric pipeline.
ShapeGen operates in two key stages. First, it builds a 'Shape Library' by training spatial warpings between 3D models. These warpings map points that serve the same function (like a handle or a spout) across different objects, creating a plug-and-play repository of shape knowledge. In the second 'Function-Aware Generation' stage, the system leverages this library to synthesize entirely new, physically plausible robotic demonstrations. It requires only minimal human annotation, such as a single demonstration on one object, to generate a multitude of variations.
The paper's real-world experiments demonstrate that policies trained with ShapeGen-generated data show markedly improved in-category shape generalizability. This means a robot trained to pour from one pitcher could successfully pour from a wide array of pitchers with different shapes, sizes, and handle positions that it never encountered during its initial training. The method sidesteps the need for complex physics simulators, offering a more direct and scalable path to creating robust robotic manipulation policies capable of operating in unstructured environments.
- Automates creation of shape-diverse robotic training data through a two-stage, simulator-free pipeline.
- Builds a 'Shape Library' of 3D models with functional warpings to map corresponding parts across objects.
- Boosts real-world robot performance, enabling generalization to unseen objects within the same category.
Why It Matters
This accelerates robot learning for real-world tasks, moving us closer to versatile robots that can handle the infinite variety of everyday objects.