LLM agents match human experts in phenotype ontology curation

A new arXiv paper (arXiv:2605.28965) from James Balhoff and Hilmar Lapp tackles the bottleneck of phenotype annotation—the labor-intensive process of linking free-text morphological descriptions to standardized ontology terms. This task is critical for cross-study integration of comparative morphology data but has traditionally required highly trained human curators. The authors revisited a 2018 benchmark that compared human curators and the Semantic CharaParser NLP tool, finding that machine-human consistency was significantly lower than inter-curator consistency.

This time, they deployed five frontier LLMs from Anthropic (Claude models) and OpenAI (GPT models) as autonomous 'agentic curators.' Each agent worked in a self-contained workspace with the original publication PDF, annotation guide, and four project ontologies (UBERON, PATO, BSPO, GO) plus a validation script. Against the same Gold Standard, every LLM agent performed within the range of inter-curator variability of the three human biocurators. The best agents approached (but did not surpass) the top human curator and substantially outperformed Semantic CharaParser across all four evaluation metrics. This demonstrates that frontier LLMs can now automate ontology curation at a level comparable to expert humans, potentially scaling phenotype data integration across thousands of studies.