Can Coding Agents Be General Agents?

A new research paper from authors Maksim Ivanov, Abhijay Rana, and Gokul Prabhakaran investigates a critical question in AI automation: whether coding agents, like those built on models such as GPT-4 or Claude 3, can evolve into general-purpose agents capable of end-to-end business process automation. Published on arXiv, the study moves beyond standard software engineering benchmarks to conduct a practical case study within a real-world open-core Enterprise Resource Planning (ERP) system. This environment tests an agent's ability to understand complex business domain logic—like inventory management or order processing—and translate it into functional code.

The findings reveal a significant capability gap. While the coding agent demonstrated reliable performance on simple, well-defined tasks, it consistently failed on more complex, multi-step business processes. The researchers identify the core bottleneck as the agent's inability to effectively bridge high-level domain-specific requirements with low-level code execution. This suggests that current coding agents lack the deep contextual understanding and reasoning required for true generalization beyond their training on code syntax and common programming patterns. The study calls for new evaluation frameworks and architectural approaches to overcome this hurdle.