Unicode Audit Uncovers Data Loss in PubMed and OpenAlex APIs
Biomedical text mining corrupted by invisible character substitution.
A new pre-registered study by Przemysław Czuma audited how faithfully four major biomedical bibliographic APIs—PubMed E-utilities, Crossref, OpenAlex, and Semantic Scholar—preserve Unicode characters from published abstracts. Using PubMed Central's JATS XML as ground truth, the study sampled 4,000 English-language articles from 2024's Open Access subset (≈700,000 records). The results reveal striking API-dependent losses: PubMed's AbstractText field retained typographic punctuation in only 0.6% of eligible abstracts (95% CI 0.3–1.0%), due to systematic character substitution. OpenAlex completely failed to preserve special whitespace (0% fidelity, CI 0.0–0.4%), traced to inverted-index serialization.
Conversely, mathematical/scientific symbols and Greek letters were preserved over 95% across all APIs. An unexpected finding: Crossref returned no abstract for 24.6% of papers (coverage 75.4%), with Elsevier and ACS contributing 0% coverage. These silent losses have direct consequences for tokenization-sensitive bibliometrics, corpus construction for biomedical LLMs, and detection of AI-assisted writing via character-level patterns. Czuma warns that the same publisher-deposited JATS text carries different surface signatures depending on which API serves it, undermining reproducibility in digital library research.
- PubMed AbstractText preserved typographic punctuation in only 0.6% of abstracts due to character substitution.
- OpenAlex lost all special whitespace (0% fidelity) caused by inverted-index serialization.
- Crossref failed to return any abstract for 24.6% of papers, with Elsevier and ACS at 0% coverage.
Why It Matters
Hidden API character corruption compromises biomedical LLM training data and bibliometric accuracy at scale.