New method steers AI driver attention without retraining

Researchers from institutions including Lars Ullrich's team have developed a method called Inference-Time Attention Steering for Vision-Language-Action (VLA) driving models. This approach enables autonomous vehicles to dynamically redirect focus toward safety-critical objects during operation without model retraining. The technique applies a bounded additive pre-softmax attention bias to visual tokens, effectively steering the model's attention during inference.

In testing on 50 lane-change scenarios from the Physical AI World Model Synthetic dataset using Alpamayo-R1's Qwen3-VL backbone, the method achieved approximately 17cm mean displacement with lateral shifts up to 140cm. The effect proved dose-dependent, with action-relevant signals located in late layers where effectiveness increased with additional hooked layers (2.0cm for first 8 layers vs 67.6cm for all 36). The steering influenced where the model looked rather than encoding specific target behaviors, suggesting potential for safer autonomous driving interventions.