New LENS framework reveals LLMs can unlearn disinformation narratives without collapse

A new research paper from Viktoriia Makovska and George Fletcher tackles a critical safety issue: can large language models be 'unlearned' to stop repeating disinformation narratives? They propose LENS (Level-based Evaluation of Narrative Suppression), a testing protocol that evaluates unlearning across four resistance levels—direct, attributed, contrastive, and abstract. The study focuses on two real-world narratives: one framing Russia's war against Ukraine as forced by NATO expansion, and one framing the US as exploiting or abandoning Taiwan. They apply LENS to four near-12B parameter multilingual instruction models: Lapa LLM, Gemma-12B, Qwen-14B, and TAIDE-Gemma.

The team introduces the Suppression-Collapse Efficiency (SCE) score, which rewards successful suppression of target narratives while penalizing degraded model output. Results show that selected checkpoints can reduce narrative reproduction, and that suppression may transfer beyond direct forget prompts. However, a surprising side effect emerged: abstract prompts (e.g., A/B/C comparisons) sometimes caused the model to recover the real-world actors associated with the target frame, effectively undoing the unlearning. The authors conclude that LENS is a successful diagnostic protocol for the deeper structure of narrative unlearning, balancing between full suppression and model collapse.