World of Code Identity Resolution Hits 99% AUC at Scale

In a new arXiv paper, Audris Mockus tackles the massive challenge of author identity disambiguation in the World of Code (WoC) dataset, which contains ~107 million distinct author strings from ~6 billion commits. Unlike previous work that focused on missed merges, Mockus identifies over-merge as the central problem at this scale, where bots, role accounts, and multi-author fields create million-member mega-clusters. The solution combines a betweenness centrality cut on the union graph with a per-edge classifier trained on 2.6 million labels mined from GitHub no-reply identifiers. The classifier, which filters dormant cross-project shingle groups and incorporates GitHub account assertions, recovers recall lost during precision work.

Results are striking: the largest cluster drops from 170,431 (and a predecessor's 3.0 million) to under 7,000, while gold recall rises from 0.44 to 0.70 at increasing precision. The per-edge model transfers with AUC 0.99 on human-adjudicated pairs, and on an independent 21-million-alias GitHub ground truth, the final map surpasses existing global, privacy-preserving resolvers. The paper also catalogs scale lessons, noting that structural cuts don't transfer to unseen edge sets and that recall-only or precision-only benchmarks invert verdicts unless read together.