LLMs lower barrier to documentation standards for agent-based modelling
Four LLMs tested for extracting RAT-RS reports from published ABM paper
Researchers Peer-Olaf Siebers and Christopher Frantz conducted a feasibility study on using Large Language Models (LLMs) to automate documentation standard adoption in Agent-Based Modelling (ABM). ABM relies on clear documentation for credibility, but standards like ODD, TRACE, and RAT-RS are underused due to the effort required. The team tested four unnamed LLMs to extract reports from a published ABM paper using the Rigour and Transparency Reporting Standard (RAT-RS).
Results showed LLMs generate coherent outputs overall, but performance varies by question type: they are more reliable on descriptive tasks (e.g., stating model components) than on explanatory or evaluative ones (e.g., justifying parameter choices). The authors identify practical heuristics for when LLM-assisted documentation is reliable and call for community-level exploration to enhance rigour and adoption in ABM reporting.
- Four LLMs were tested for extracting RAT-RS documentation from a published ABM paper
- LLMs performed better on descriptive tasks than explanatory or evaluative ones
- Human oversight remains necessary for reliable documentation extraction
Why It Matters
Automating documentation lowers barriers to standard adoption, improving transparency and credibility in agent-based modelling research.