Can Large Language Models Implement Agent-Based Models? An ODD-based Replication Study

A new study tested 17 large language models on their ability to generate executable, scientifically valid code for complex agent-based simulations from a standardized specification. GPT-4.1 consistently produced statistically valid and efficient Python implementations of a predator-prey model, while Claude 3.7 Sonnet performed well but less reliably. The research found that while faithful implementations are possible, executability alone is insufficient for reliable scientific use, highlighting current limitations.