NYU-EPFL team debuts VR system for human-guided drone team navigation

A team led by researchers from New York University (NYU) and the Swiss Federal Institute of Technology Lausanne (EPFL) has unveiled "Flying Together", a Virtual Reality (VR) shared control framework that lets a human operator intuitively guide multiple drones through unknown, constrained spaces. Published as arXiv:2605.21680 and accepted at IEEE ICRA 2026 (Vienna), the system addresses a critical weakness of fully autonomous multi-robot systems: their inability to adapt to unforeseen conditions or capture operator-driven objectives in unstructured environments.

The core innovation is a user-guided motion-primitive-based planner that computes continuous, collision-free trajectories while continuously blending in operator input via an admittance controller. This allows the human to flexibly steer drones toward regions of interest that autonomous planners might miss. The framework supports mixed-reality setups with both physical drones and simulated agents, and features a bilateral VR interface where the operator guides the team through "migration points" and receives immediate visual feedback on team state. Experiments demonstrated that the shared control approach significantly improves obstacle avoidance, maintains proper inter-agent spacing, and cuts operator effort compared to fully autonomous or fully manual control. The work offers a practical path for deploying drone swarms in real-world search-and-rescue, inspection, or exploration missions where human judgment remains essential.