Oxford study: Open-weight models beat GenAI for keyword extraction in crowdsourced archives

A new paper from Oxford researchers (Arana-Catania, Conisbee, Kidd) tests three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modeling—on the 'Their Finest Hour' Second World War crowdsourced archive. The study spans traditional statistical methods to modern GenAI neural networks, concluding that no single technique works universally. Open-weight, extractive models (like those from Hugging Face) emerged as the safest bet for responsible deployment, offering transparency and control over metadata generation.

Generative AI, meanwhile, worried the team due to 'accountability risks'—its abstractive nature can alter meaning in ways hard to trace, especially when the metadata directly ties to living contributors. The 45-page report with 6 tables argues that in crowdsourced collections, stewardship duties must match technical performance. The findings are a practical guide for any institution scaling metadata labeling with AI, emphasizing humans-in-the-loop and model choice as critical to ethical crowdsourcing.