GeoDrive-Bench exposes VLMs' weak regional driving IQ across 6 countries
New benchmark tests 9 top VLMs on 5,053 region-specific driving scenarios—results vary wildly.
A new research paper from Yingzi Ma, Chaowei Xiao, and Ming Jiang introduces GeoDrive-Bench, a benchmark designed to evaluate how well vision-language models (VLMs) handle region-specific traffic rules and driving cultures. The benchmark comprises 5,053 multiple-choice QA pairs, each validated by human annotators, covering six countries with distinct driving conventions. It focuses on four core autonomous driving tasks: perception (identifying objects and signs), prediction (anticipating other road users' behavior), planning (choosing driving actions), and region reasoning (inferring local traffic norms from visual cues without explicit country labels).
When tested on nine state-of-the-art VLMs, GeoDrive-Bench exposed substantial performance gaps across different geo-driving cultures. The authors also proposed a distillation algorithm that injects region-specific traffic-rule knowledge into the internal representations of VLMs, enabling them to better align visual scene understanding with local policies. Their baseline models showed improved geo-cultural reasoning, but overall results suggest that current VLMs still lack robust region-aware driving intelligence. GeoDrive-Bench thus serves as both a diagnostic tool and a training-oriented testbed for building deployable autonomous driving foundation models that can safely operate across diverse global environments.
- Benchmark includes 5,053 human-validated QA pairs across six countries (US, China, Germany, etc.) covering perception, prediction, planning, and region reasoning tasks.
- Nine state-of-the-art VLMs showed significant performance variation across geo-driving cultures, indicating lack of robust region-aware intelligence.
- A distillation algorithm was developed to inject local traffic-rule knowledge into VLM internal representations without needing explicit country labels.
Why It Matters
Ensuring autonomous driving systems understand local traffic rules is critical for safe global deployment; this benchmark reveals a major blind spot.