HANDFUL: New AI teaches robot hands to conserve fingers for multi-step tasks

Researchers from the University of Southern California and Amazon have developed HANDFUL, a novel learning framework that enables dexterous robot hands to perform sequential manipulation tasks while intelligently managing finger usage as a limited resource. The work, published on arXiv, addresses a key gap in robotics: most prior research focuses on single-object, single-skill tasks, but real-world applications often require a robot to execute multiple skills in sequence while maintaining control over previously grasped objects. HANDFUL introduces a resource-aware approach where the robot learns to grasp objects in a way that conserves fingers for subsequent actions, such as pushing, pulling, or pressing a second object.

The framework uses finger-level contact rewards to encourage exploration of grasps that leave fingers free for later tasks, combined with curriculum-based policy learning to select optimal grasps for downstream objectives. The team also created HANDFUL-Bench, a simulation benchmark with multiple sequential dexterous manipulation tasks. Extensive experiments show that prioritizing resource-aware grasps significantly improves second-subtask success and robustness compared to a baseline that greedily optimizes the initial grasp. The approach was validated on both simulated environments and a real LEAP dexterous hand, demonstrating real-world applicability. This work establishes resource-aware grasp planning as a key principle for multifunctional dexterous manipulation, with potential applications in manufacturing, healthcare, and household robotics.