Adversarial Flow Matching for Imperceptible Attacks on End-to-End Autonomous Driving

Autonomous driving is increasingly adopting end-to-end (E2E) frameworks, either as monolithic Vision-Language-Action (VLA) models or specialized modular architectures. Both rely heavily on Transformer backbones for reasoning, creating a shared vulnerability: visually imperceptible perturbations can hijack these models into dangerous maneuvers. Existing adversarial attacks (white-box or black-box) either demand full model transparency or suffer from high latency and limited transferability. In a new paper on arXiv (2605.00880), Xinyu Zeng and colleagues introduce Adversarial Flow Matching (AFM), a gray-box framework that exploits Transformer structural weaknesses without requiring complete model access.

AFM uses a neural average velocity field to generate adversarial examples in a single step, perturbing both the generative latent space and the velocity field for optimal imperceptibility. Experiments across various driving scenarios show AFM substantially degrades performance in both VLA and modular agents compared to baselines, while achieving state-of-the-art visual quality. Crucially, the generated attacks transfer robustly between different Transformer-based models, meaning AFM functions almost like a black-box attack—only the prior knowledge of a Transformer module is needed. This work highlights a critical security blind spot in modern E2E autonomous driving pipelines.