V-Steer boosts LLM instruction safety from 18% to 92% without retraining

LLMs often fail to respect instruction hierarchies, where system prompts should override conflicting user or tool inputs. This is a critical safety gap. Researchers from UIUC present V-Steer, a training-free inference-time method that restores privileged influence by directly editing cached value vectors at prompt positions. Using direct logit attribution on the first next token prediction, V-Steer identifies attention heads where lower-priority spans dominate, then boosts privileged spans and suppresses conflicting ones through in-place multiplicative edits. Since it only acts on cached values, V-Steer remains compatible with fused attention backends and adds only a one-time prefill overhead.

Across models from 7B to 70B parameters, V-Steer raises primary constraint accuracy from under 18% to 92% on controlled role conflict benchmarks. On broader instruction hierarchy evaluations, it substantially outperforms prompt-only baselines and matches or exceeds state-of-the-art training-based methods on 3 of 4 model scales, all with negligible decoding-speed overhead. The method is published at COLM '26 and code is available. This approach offers a practical, cost-effective way to enforce safety constraints in deployed LLMs without retraining.