Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models

A new paper from Shubham Kumar and Narendra Ahuja at the University of Illinois introduces LOCA (Local, Causal Explanations), a method that pinpoints exactly which internal model changes cause a jailbreak to succeed. Unlike prior work that looked for global concept directions (like “harmfulness” or “refusal”) and tried to explain all attacks the same way, LOCA treats each jailbreak attempt individually. It identifies a minimal set of interpretable intermediate representation changes that, when applied to the model's forward pass, causally induce refusal on an otherwise successful harmful request. In tests on Gemma and Llama chat models using a large jailbreak benchmark, LOCA needed only six content edits on average to flip a successful attack into a refusal. In contrast, earlier methods like activation steering or probing-based interventions routinely failed to achieve refusal even after 20 changes.

The implications for AI safety are significant. Current safety training creates a fragile barrier that can be bypassed by carefully crafted prompts. LOCA reveals that different jailbreak strategies succeed by manipulating different internal concepts, and that the same strategy may not work across different harmful categories (e.g., violence vs. cyber-attacks). By offering local explanations, LOCA moves beyond the one-size-fits-all understanding of jailbreaks and opens the door to more targeted defenses. The authors plan to release their code, which will allow other researchers to apply LOCA to their own models and jailbreak datasets. This work is a critical step toward robust, mechanistic safety guarantees for future autonomous AI systems that operate in high-stakes environments.