Joint Velocity Action Spaces Outperform in Robotic Manipulation RL
New benchmarking study reveals joint velocity as the optimal action space for vision-based robotic tasks.
A new study from researchers at the University of Alberta and the National Research Council Canada benchmarks four action space representations in reinforcement learning (RL) for vision-based robotic manipulation: pose increment, pose velocity, joint position increment, and joint velocity. The team trained RL policies in simulation and transferred them to real-world robots using sim-to-real transfer, focusing on two common tasks: object picking and object pushing.
The results reveal that the joint velocity action space significantly outperforms the alternatives in both motion smoothness and final task performance. The authors recommend joint velocity for practitioners working on similar vision-based manipulation tasks. They also provide practical guidelines on selecting action spaces for both simulation and real-world experiments, emphasizing that action space choice can make or break sim-to-real transfer success.
- Joint velocity action space produced the smoothest motions and highest task performance for both picking and pushing tasks.
- Four action spaces were compared: pose increment, pose velocity, joint position increment, and joint velocity in sim-to-real transfer.
- The study offers practical guidance for RL practitioners on choosing action spaces for real-world robotic manipulation.
Why It Matters
Choosing the right action space can dramatically improve real-world robot performance, reducing jerky motion and task failures.