arXiv study: 89% of biomedical papers now show LLM-assisted writing

A new preprint from arXiv (2608.10715) by researchers Lena Holzwarth, Rita González-Márquez, and Dmitry Kobak presents an unbiased approach to detect LLM-assisted writing in scholarly publications. Rather than relying on LLM output detection tools, which can be unreliable, the team analyzed shifts in word frequency distributions—a signal that captures the subtle vocabulary changes introduced by AI writing assistants. They validated the method on known corpora, then applied it to the full text of open-access biomedical papers from PubMed Central.

The results are striking: by the end of 2025, 89% of biomedical papers showed a statistically significant excess of LLM-associated vocabulary compared to pre-LLM baselines. The study also reveals a clear section-level bias. LLM usage is twice as likely in Discussion paragraphs (68%) than in Methods paragraphs (32%), likely because Discussion sections require broader, more subjective language that aligns naturally with LLM strengths. Yet even in the Methods section—often considered the most factual and structured part of a paper—over half of all paragraphs showed signs of LLM assistance.

The authors argue their word-frequency approach is more robust than existing detectors because it doesn't overfit to specific LLM outputs and can scale across millions of documents. They emphasize that their estimates are not about policing individual authors but about monitoring the systemic shift in academic writing. As LLMs become embedded in the writing process, from non-native English speakers to professional researchers, the boundaries of authorship and originality blur.

The paper's findings have immediate policy implications. Journals, funding agencies, and academic institutions are already debating how to handle LLM disclosure. With the evidence that LLM-assisted writing has become the default rather than the exception, the conversation shifts from "should we ban it" to "how do we govern it transparently." The researchers hope their method provides a baseline for evidence-based policy development, ensuring that governance matches the reality of modern scientific publishing.