Structural diversity metric predicts disruptive science across 260M papers

A new paper from Peng et al., posted on arXiv, introduces Structural Diversity (SD) as a powerful predictor of disruptive scientific innovation. SD measures the extent to which a team’s prior collaboration network connects multiple distinct knowledge communities—going beyond simple metrics like team size or demographic diversity. Using a century-scale dataset of 260 million scientific publications from 1900 to 2025, combined with causal inference methods and a quasi-natural experiment based on a 2012 U.S. National Science Foundation policy change, the authors demonstrate that SD consistently outperforms traditional indicators such as team freshness and edge density in forecasting breakthrough research.

Notably, SD positively interacts with team size, mitigating the well-known “curse of scale” that often plagues larger collaborations. Instead of diluting impact, larger teams with high SD actually amplify creative synthesis through a mechanism called Disciplinary Integration (DI)—the ability to combine heterogeneous knowledge into novel configurations. The findings position SD as both a theoretical construct and an actionable design principle: organizations can systematically engineer collaborative structures to foster disruptive innovation by ensuring teams bridge diverse intellectual communities.