New study finds LLM conformity is mostly just repeated text, not social influence

A new paper from Yibo Hu and Jiaming Qu (arXiv 2607.05545) challenges the common interpretation of LLM conformity—where models change correct answers to match a peer or group. The authors find that most of this apparent conformity is actually due to the repeated wrong answer itself, not social influence. They introduce a 'no-source condition' that removes the speaker while keeping the repeated answer. Across six open-weight LLMs and seven QA/reasoning datasets, this condition alone caused harmful revision in 66.5% of initially correct cases, compared to just 10.3% under a plain re-ask. The effect persists even when the wrong answer is paraphrased or answer options are hidden (open-ended).

Source framing does modulate this floor: referring to an 'expert panel' raises the revision rate, while minimal person labels (e.g., 'Person A') do not reliably increase it further. When models flip, they are usually confidently wrong, and simple recalibration fails to recover the original answer. The methodological takeaway is that conformity benchmarks should first measure the speaker-free floor—the revision rate from repeated text alone—before attributing any change to social pressure. Without this step, benchmarks may confuse repetition with conformity. The paper provides a crucial nuance for evaluating LLM behavior in multi-turn or group settings.