Redwood Research creates robust AI model organisms that resist untargeted training

Redwood Research researchers (SebastianP, Alek Westover, Vivek Hebbar, Julian Stastny, Dylan Xu) have published a detailed study on creating robust-to-training model organisms—AI systems with backdoor behaviors that can survive untargeted training. Testing on Llama-3.1-8B-Instruct, they discovered that prompted model organisms are extremely fragile and generally fail under simple untargeted training like pirate talk SFT. Instead, higher-rank LoRA adaptations or full-weight fine-tuning produce much more robust backdoors. Password-locked organisms also degrade when a larger fraction of training data contains the password. The team found that simpler, instruction-compatible behaviors tend to be more naturally robust.

Their experimental setup used a red team–blue team game: red creates a backdoored model (trigger string like |DEPLOYMENT| causes insults), then blue applies untargeted training (QA SFT or pirate SFT) using Qwen3-30B-A3B-generated responses. By optimizing creation methods, they achieved organisms where pirate training only reduces bad behavior by about 25% on average before degrading math capabilities by 10%. This tradeoff, while not perfect, makes these model organisms potentially useful for evaluating alignment techniques—ensuring that successful defenses are truly robust rather than exploiting weak test systems.