Research reveals 84% of software citations are inconsistent

Researcher Pengyin Shan published a comprehensive audit of software citation metadata consistency across 117 open-source research projects, revealing systematic discrepancies that could fragment academic credit and provenance. The study analyzed up to seven machine-readable surfaces per project—including citation files, archive deposits, DOI records, and package registries—using a four-level verdict rubric across six metadata fields. With 98.5% hand-verified precision, the audit found that 83.9% of projects with multiple comparable surfaces contained at least one core-field conflict, a finding insensitive to fuzzy-matching thresholds.

The research identified a critical mechanism behind 50% of conflicts: surfaces often describe the software's associated paper rather than the software itself. Author lists and titles showed the highest disagreement rates, while registry surfaces were the least aligned. Notably, 28 of 32 projects with a preferred citation URL routed readers to records that conflicted with the software's own metadata. To enable reproducibility, Shan released the audit pipeline as an importable library, along with the full corpus, sampling protocol, raw snapshots, and verification logs.