Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control

MIT researchers (Ho Jae Lee, Sangbae Kim et al.) have published a paper introducing a hybrid hierarchical control framework for reactive dexterous grasping. The system separates high-level planning from low-level control using a multi-agent reinforcement learning (RL) architecture—with distinct arm and hand agents—that generates task-space velocity commands. A GPU-parallelized quadratic programming (QP) controller then translates these into feasible joint velocities while strictly enforcing kinematic limits and collision avoidance. This structural isolation accelerates training convergence and ensures hardware safety.

Extensive simulation-to-reality validation on a 7-DoF arm paired with a 20-DoF anthropomorphic hand demonstrates robust zero-shot transferability to a diverse set of unseen objects. The system reactively recovers from unexpected physical disturbances in unstructured environments. A key innovation is zero-shot steerability: operators can dynamically adjust safety margins and avoid dynamic obstacles without retraining the policy. This work advances the practicality of dexterous manipulation in real-world, dynamic settings.