EnergyAction: Unimanual to Bimanual Composition with Energy-Based Models

A research team led by Mingchen Song has introduced EnergyAction, a breakthrough framework that enables robots to perform complex two-handed tasks by combining pre-trained single-arm skills. The core innovation uses Energy-Based Models (EBMs) to mathematically compose individual left and right arm policies into coordinated bimanual actions. This approach solves a major challenge in robotics: while abundant data exists for single-arm manipulation, collecting demonstrations for two-handed coordination is extremely difficult and time-consuming.

EnergyAction incorporates three key technical innovations. First, it models individual unimanual policies as EBMs, leveraging their compositional properties to fuse left and right arm actions. Second, it introduces energy-based temporal-spatial coordination constraints that ensure generated actions are both temporally coherent and spatially feasible. Third, the framework implements two energy-aware denoising strategies that dynamically adapt denoising steps based on action quality assessment, maintaining computational efficiency while ensuring high-quality output.

Experimental results demonstrate that EnergyAction effectively transfers knowledge from single-arm to dual-arm manipulation, achieving superior performance on both simulated and real-world tasks with minimal bimanual data. The framework represents a significant advancement in sample-efficient robotics learning, potentially accelerating the development of robots capable of complex manipulation tasks in unstructured environments like warehouses, kitchens, and manufacturing facilities.