VERAGMIL uses VR demos to train feeding robots with near-human scooping

Robot-assisted feeding (RAF) systems aim to help individuals with disabilities or motor impairments eat independently, but manipulating granular foods like rice and beans remains a major challenge due to their unpredictable physical behavior. Traditional programming approaches struggle with these dynamics, so researchers often turn to imitation learning. However, collecting high-quality human demonstrations for such tasks is time-consuming and difficult. To solve this, Amanuel Ergogo, Diego Dall'Alba, and Przemyslaw Korzeniowski developed VERAGMIL, a framework that pairs a high-fidelity physics simulator with an intuitive Virtual Reality (VR) interface. This setup lets humans demonstrate scooping and transporting motions naturally, while the system records rich data for training several imitation learning models—specifically behavior cloning (BC), a recurrent variant (BC-RNN), and batch-constrained Q-learning (BCQ).

In their evaluation, the team trained these models on granular scooping and transport tasks using both VR demonstrations and a 3D space mouse, then compared them against a human-expert baseline across success rate, spillage, generalization to unseen food items, and completion time. The results were clear: VR-based demonstrations significantly outperformed 3D space mouse data across all metrics, showing that the quality of the demonstration interface matters as much as the algorithm. Among the models, BCQ was the standout, achieving the best overall performance and specifically reducing spillage to a level that closely approached human skill. This work, published at IROS 2025, shows that combining realistic simulation with VR demonstration collection can effectively train robust policies for handling granular materials. By open-sourcing the code, the team gives other researchers a practical platform to accelerate progress in assistive robotics and bring reliable robot feeding closer to real-world deployment.