LLM Stance Simulation Easily Manipulated by Context Changes, Study Finds

A new preprint on arXiv (2606.06443) by Xinnong Zhang, Wanting Shan, Hanjia Lyu, Zhongyu Wei, and Jiebo Luo investigates how sensitive LLM-based stance simulations are to changes in conversational context. The researchers used a counterfactual framework: given an online discussion, they first inferred a target user's stance using an LLM, then applied controlled revisions to the context (e.g., rephrasing replies, adding memes) and re-simulated the stance. They measured directional stance shift and stance transition rate, comparing text-only revisions against a multimodal version that incorporated memes.

The results show that both text-only and multimodal context revisions can cause significant stance changes, even when the semantic content is largely independent of the original discussion. This highlights that LLMs do not simply reflect fixed user beliefs but are highly context-sensitive. The authors warn that using LLMs to simulate online opinion dynamics—common in social science research—may produce unreliable conclusions if contextual subtleties are not controlled. The work serves as an audit framework and underscores both the promise and ethical risks of deploying LLMs for stance simulation and digital twin modeling.