New study finds language follows 1/d scaling law for context

In a paper posted on arXiv (2607.25184), researcher Elan Barenholtz presents a scaling law of contextual persistence in human language. Using large language models as probabilistic probes, he measured the reduction in target word perplexity conferred by prior context at distance d, beyond that of the same words scrambled. This difference, called the contextual persistence function P(d), isolates the influence of word arrangement. Across ten corpora spanning six language families and both written and spoken modalities, P(d) decays approximately as 1/d, following P(d) ∝ d⁻α with a mean α = 1.04 and median r² = 0.96. The effect vanishes in scrambled and synthetic controls, is replicated across independent probes, and does not appear in genomic or protein sequences under domain-native models.

An exponent near 1 distributes contextual influence roughly uniformly across logarithmic timescales, meaning that words far apart in a sentence still exert meaningful statistical influence on each other. This mirrors other known scaling laws in language (e.g., word frequency, vocabulary growth) but applies specifically to sequential arrangement—a central determinant of meaning. The finding suggests a fundamental, possibly universal, property of human language structure, with implications for natural language processing models, cognitive science, and our understanding of how context shapes comprehension. Barenholtz's work establishes a new quantitative law for the field.