AgileX's Isaac Lab pipeline automates block stacking data generation

AgileX Robotics has released an upgraded teleoperation and data collection pipeline for Isaac Lab that automates block stacking demonstration generation. Building on their previous manual keyboard-teleoperation workflow, version 2.0 introduces a fully autonomous controller capable of executing pick-and-place tasks with precise Cartesian pose increments. The system combines cube position detection, a trajectory planner using PCHIP smoothing, and an IK stack controller that interfaces with Isaac Lab's Differential IK module. The pipeline iterates over multiple cubes, sequentially lifting and placing them to form a stack, while automatically detecting success and recording trajectories into HDF5 files.

This new approach eliminates the need for human operators during data collection, producing consistent, scalable datasets for robot learning. The controller outputs action vectors (Δx, Δy, Δz, Δrx, Δry, Δrz, gripper) rather than joint angles, making it reusable across different robot configurations. The entire sequence—approach, descend, grasp, lift, move to target, place, retreat—runs without interruption, with each step's duration and gripper state predefined. The resulting pipeline is ideal for researchers and developers building imitation learning or reinforcement learning systems for manipulation tasks, significantly reducing the time and effort required to gather expert demonstrations.