Researchers release PandasCorpus dataset for 4M Pandas API calls
139k Jupyter notebooks analyzed revealing Pandas usage patterns from 100k repos.
Researchers Syrym Abdikhan and Mazhar Hameed have released PandasCorpus, a first-of-its-kind dataset designed to dissect real-world Pandas workflows at scale.
The corpus aggregates data from 139,000 Jupyter notebooks across approximately 100,000 GitHub repositories, capturing over 4 million Pandas API calls spanning 136 distinct operations. By analyzing structural and Pandas-specific features, the study examines code executability, notebook size trends, and recurring operation sequences from 2015 to 2025. This dataset not only reveals empirical patterns in data analysis workflows but also provides a reusable resource for library-aware code composition research. Both the dataset and extraction pipeline are publicly accessible via GitHub and Zenodo, enabling further academic and industry research into Pandas usage patterns.
- PandasCorpus includes 4M+ API calls from 139k Jupyter notebooks in 100k GitHub repos
- Dataset covers 136 Pandas operations and tracks workflow evolution (2015-2025)
- Both dataset and extraction pipeline are open-source on GitHub and Zenodo
Why It Matters
Empowers data scientists to optimize Pandas workflows and improve tooling based on real usage patterns.