Google's Gemma-2 models reveal new insights into AI conflict resolution
Google's Gemma-2 and Pythia models expose how AI systems handle conflicting rules vs defaults
A new study published on arXiv explores how large language models (LLMs) handle conflicting instructions by examining 'congruency effects'—a phenomenon well-documented in human psychology where responses are influenced by competing cues. Researchers from the University of Michigan and collaborators tested Google's open-source Gemma-2-2B model alongside six Pythia models ranging from 410M to 12B parameters in a verbal conflict task. The task required models to either follow a default same-color response or adhere to an explicit rule that sometimes conflicted with the default.
The findings reveal that all tested models exhibited strong default response tendencies, and all but one showed significant congruency effects—meaning their behavior was heavily influenced by whether the rule agreed with or contradicted their default tendencies. Using techniques like causal attribution analysis and attention ablations, the team identified two distinct processing pathways: one preferentially activated when rules align with defaults (short-range attention to superficial cues) and another engaged when rules conflict with defaults (long-range attention to the rule prefix). Fine-tuning the models to strengthen default tendencies improved performance in congruent scenarios but degraded it in incongruent ones, while increasing rule set size selectively impaired incongruent performance. These results suggest that congruency effects in LLMs arise from competition between 'in-weight' (learned) default mappings and 'in-context' (rule-based) mappings, offering a mechanistic explanation for how LLMs resolve conflicting information within a single network.
- Tested Google's Gemma-2-2B and six Pythia models (410M–12B parameters) in a verbal conflict task with default vs. rule-based responses.
- Models showed strong congruency effects, favoring defaults unless explicitly overridden by rules, with distinct attention pathways for congruent/incongruent conditions.
- Fine-tuning defaults improved congruent performance but hurt incongruent performance, while larger rule sets selectively impaired incongruent responses.
Why It Matters
Unlocks new ways to predict and control AI behavior in conflicting scenarios, critical for safety-critical applications like autonomous systems and medical diagnostics.