125 Wikipedia edits by advocates measurably shape LLM outputs on animal welfare

A new arXiv paper (arXiv:2606.24890) demonstrates that a tiny group of coordinated Wikipedia editors can significantly shape how large language models discuss sensitive topics. The Pro-Animal Wikipedians (PAW) made just 125 edits across 115 pages, adding sourced animal welfare content. Using gradient-based data attribution methods (Bergson and MAGIC) and TrackStar retrieval attribution on Llama 3.1 8B, the researchers found that PAW-edited sections constituted 68% of the highest-attributed documents for animal welfare queries (p < 0.0001), compared to only 52% for unrelated queries about the same companies (p = 0.53). This indicates the model specifically links the edits to animal welfare topics, not general entities.

Counterfactual influence estimation on Llama-3.2-1B across five random training-order seeds showed an even sharper effect: in every seed, the top-10 most influential documents on animal welfare queries were all PAW edits (10 of 10, 5 of 5 seeds), while on general queries the same top-10 sat at chance (4–6 of 10). Leave-subset-out validation gave Spearman rho = 1.00 across all runs. When the team fine-tuned separate models on PAW content vs. control content, each model performed better on its training type: the PAW-trained model cut perplexity on animal welfare text from 12.4 to 8.4, while the control model cut control-text perplexity from 16.1 to 11.4. These findings reveal that even small, coordinated Wikipedia editing campaigns can measurably influence how language models handle the topics those edits address, raising important questions about the reliability and neutrality of LLMs built on crowd-sourced data.