Why AI Safety's 'Model Organisms' Approach Could Be Leading Researchers Astray

J Bostock, writing for Arcadia Impact's Alignment Team on LessWrong, delves into the metaphor of 'model organisms' borrowed from biology and applies it to AI safety research. In biology, a model organism like Mus musculus (the lab mouse) is chosen for practicality and extensive prior knowledge, enabling researchers to generalize findings to other species. Bostock highlights that this generalization depends on shared evolutionary ancestry and similar objectives. He warns that convenience and understanding create a feedback loop, but also risk over-reliance on a narrow set of models, potentially undermining generalizability to other AI systems.

Turning to AI safety, Bostock distinguishes three research categories: studying a production language model to infer general behavior, studying a model with a specific intervention to prove its effects, and studying a model with a particular property to make inferences about that property in other models. He echoes Francis Rhys Ward's taxonomy and emphasizes the need for clarity. Drawing parallels to biological knockouts and disease models, Bostock argues that AI researchers must carefully justify why their chosen model represents a broader class, just as biologists rely on common ancestry. The article calls for rigorous methodology to avoid conflating idiosyncratic findings with universal truths.