Drift and selection in LLM text ecosystems

A new research paper by Søren Riis, 'Drift and Selection in LLM Text Ecosystems,' provides a mathematical model for a critical problem in AI development: the recursive feedback loop where text generated by models like GPT-4 and Claude enters the public record and is later used to train their successors. The paper develops an exactly solvable framework using variable-order n-gram agents to analyze this self-referential system. It separates two key forces acting on the corpus: 'drift,' where unfiltered reuse progressively strips out rare and novel linguistic forms, and 'selection,' where human or algorithmic filters (like publication standards or fact-checking) decide what enters the record.

Riis's analysis reveals a stark divergence in outcomes based on the type of selection applied. If publication merely reflects the statistical status quo—publishing what is already common—the entire text ecosystem converges to a 'shallow' equilibrium where language loses depth and further predictive lookahead provides no benefit. However, if selection is 'normative,' actively rewarding qualities like correctness, novelty, or high quality, richer linguistic structure can persist. The paper establishes an optimal upper bound on how much divergence from shallow states this normative filtering can sustain. This framework moves the discussion from anecdotal concern to a formal, analyzable model of data degradation.

The implications are direct for companies building AI training pipelines, such as OpenAI, Anthropic, and Meta. The research provides a tool to identify when recursive data collection is compressing and impoverishing the textual universe versus when selective filtering can maintain a healthy, evolving ecosystem. It argues that the design of curation mechanisms for massive corpora is not just a practical concern but a foundational one for the long-term capabilities of AI systems, pushing for intentional, quality-focused data selection strategies over purely scale-driven approaches.