AirForesight teaches drones to 'imagine' future maps for language-guided navigation

Unmanned Aerial Vehicle Vision-Language Navigation (UAV-VLN) requires drones to interpret natural language instructions, infer spatial structure from limited multi-view images, and execute safe 3D movements in outdoor settings. Existing approaches often map vision-language inputs directly to actions, leaving little room for explicit scene understanding or foresight. AirForesight, a new framework from a team including Yutong Liu and colleagues, tackles this by learning a structured current-map representation from sparse observations. This representation is jointly trained with two objectives—reconstructing the present scene and predicting future trajectories—so it encodes both the current layout and the intended motion path.

The framework then uses structured causal attention to propagate this current spatial knowledge into future-map reasoning, aggregating both to predict the next 3D waypoint. A novel cross-space planning consistency loss aligns the predicted map-space trajectory with the expert action direction derived from ground-truth waypoint displacement, making the imagined maps more navigation-relevant. Experiments on the OpenUAV and AerialVLN-S benchmarks show strong performance gains and stable training, validating the design. Accepted at ACM Multimedia 2026, AirForesight moves beyond reactive mapping by giving drones a way to anticipate what comes next—a critical step for autonomous drone delivery, infrastructure inspection, and search-and-rescue missions.