AI Simulations of Human Behavior Are Sensitive to Your Exact Wording

Researchers ran a simulated epidemic where AI agents woke up each day and decided whether to isolate or socialize. The AI was given a prompt describing the situation. The researchers wanted to see if tiny differences in how the prompt was phrased would change the agents' decisions. This matters because AI agents are increasingly used to stand in for humans in simulations.

When the prompt used synonyms — like saying "hunker down" instead of "isolate" — the simulation results barely changed. But when the researchers made slightly larger changes, like changing the context or adding a minor detail, the model's outcomes shifted noticeably. Interestingly, giving the agents different names (like "Alice" vs. "Bob") didn't change anything, even when the names were meant to suggest different personalities.

Companies and governments use AI to predict how people will behave during a pandemic, a natural disaster, or a new policy. If results depend on loose phrasing of the prompt, decision makers might get different conclusions just by asking in a slightly different way. That doesn't mean these models are useless, but it does mean we need to be careful and test multiple phrasings before trusting the output.

The catch: this study only tested one scenario — an epidemic model. Other types of AI simulations may be more or less sensitive. Also, the agents aren't real people; they're approximations. So we shouldn't take any single simulation as gospel. Instead, use them as a way to explore possibilities, not as a precise forecast of what humans will do.