Evaluating Digital Inclusiveness of Digital Agri-Food Tools Using Large Language Models: A Comparative Analysis Between Human and AI-Based Evaluations

A team of researchers has published a groundbreaking study exploring whether large language models (LLMs) can automate the complex evaluation of digital inclusiveness in agricultural tools. The research, led by Githma Pewinya, Carolina Martins, and Garcia Mariangel, benchmarks four leading models—Grok, Gemini, GPT-4o, and the newly released GPT-5—against the established, human-led Multidimensional Digital Inclusiveness Index (MDII). The MDII is a comprehensive framework used to assess how accessible and equitable digital agri-food tools are, particularly in the Global South, but its manual application can take months to complete.

The study's comparative analysis reveals that LLMs can generate evaluative outputs that closely approximate expert human judgment in several key dimensions. The researchers investigated model alignment with prior human scores, sensitivity to different temperature settings (which control output randomness), and potential sources of algorithmic bias. While the findings are exploratory, they provide early evidence that generative AI could be integrated into development monitoring workflows. This integration promises to scale evaluations in time-sensitive or resource-constrained environments, potentially democratizing access to critical inclusion audits.

The work highlights both the promise and the current limitations of this approach. Reliability varied significantly across the different LLMs and specific evaluation contexts, indicating that AI is not yet a perfect substitute for human expertise. However, the study establishes a crucial proof-of-concept for using AI as a supportive tool to complement, rather than replace, existing human-centric frameworks. This could lead to faster, more scalable assessments of whether digital tools truly serve diverse, often marginalized, farming communities.