Evaluating AGENTS.md: are they helpful for coding agents?
The standard practice for coding agents might be making them worse at their jobs.
Deep Dive
A new study rigorously evaluates the common practice of using AGENTS.md files to guide AI coding agents. Researchers found that providing these repository-level context files actually reduces task success rates compared to giving the agent no context at all. The files also increase inference costs by over 20%. Both AI-generated and human-written context files encourage broader exploration but add unnecessary requirements that make tasks harder for the agents to complete successfully.
Why It Matters
This challenges a core industry practice, suggesting developers should write minimal context files to maximize agent performance and cost-efficiency.