From Guidelines to Practice: Evaluating the Reproducibility of Methods in Computational Social Science

A new study tested reproducibility in computational social science across 47 sessions, finding curated documentation and preset environments dramatically improve success rates. The research revealed participants frequently used AI tools for troubleshooting, enabling independent resolution of issues without help. Standardized execution environments yielded the highest success rates and fastest completion times. The findings highlight multi-layered barriers requiring better documentation, stable environments, and conceptual clarity for reproducible science.