Tokyo Researchers Unveil VLAff: The Robot Chef That Learns from Humans

Researchers from the University of Tokyo, led by Jihoon Oh and Kento Kawaharazuka, have introduced **VLAff**, a groundbreaking vision-language-affordance model designed to bridge the gap between human demonstration videos and robot execution. The core innovation lies in its ability to extract **embodiment-agnostic actionable affordances**—visual cues indicating where to interact, how to grasp, and how to move—directly from egocentric human videos. To enable this, the team constructed **EgoAffordance**, a massive dataset containing 204K video episodes with 5.6M visual affordances and 11.6M grasp/trajectory annotations, all generated using 3D Structure-from-Motion and hand mesh reconstruction techniques.

VLAff leverages a unified foundation model architecture that correlates vision, language, and affordance data, generating executable robot actions in real-time. Given a visual input and instruction, the model outputs **affordance heatmaps, grasp poses, and motion trajectories**, which are then translated into low-level robot commands using 3D scene data. In experiments, VLAff achieved state-of-the-art performance in visual affordance prediction and demonstrated zero-shot manipulation capabilities, eliminating the need for task-specific training. The work, accepted to IEEE/RSJ IROS 2026, represents a leap toward scalable, human-like robot learning from passive observation.