Researchers unveil BioMedJImpact dataset to track AI's role in biomedical journals
A new dataset analyzes 1.74M PubMed articles to quantify AI's growing influence in biomedical research impact
Researchers have introduced BioMedJImpact, a comprehensive dataset and LLM-based pipeline designed to analyze AI's role in biomedical research impact. Built from 1.74 million PubMed Central articles spanning 2,744 journals, the dataset integrates bibliometric indicators, collaboration metrics, and an LLM-derived AI engagement rate—the proportion of AI-related articles per journal-year.
The team used a reproducible three-stage LLM pipeline to extract AI relevance and validate results through human evaluation, achieving substantial agreement. Their analysis revealed two key patterns: larger author teams consistently correlate with higher citation impact, while AI engagement positively influenced Impact Factor only in the 2019 subset. The findings, published on arXiv (arXiv:2608.05227), provide a scalable framework for tracking AI's evolving role in scientific prestige.
- BioMedJImpact dataset includes 1.74M PubMed articles from 2,744 journals with LLM-calculated AI engagement rates
- LLM pipeline validated with human evaluation, showing strong agreement in AI relevance detection
- Collaboration size correlates with citation impact, while AI engagement only boosted Impact Factor in 2019
Why It Matters
Enables data-driven decisions about AI integration in research and funding strategies for biomedical institutions