Why AI Coding Agents Keep Failing at the Worst Moment — and the New Framework That Stops It

As AI coding agents are increasingly deployed for software engineering tasks, rare but destructive failure modes emerge. In a new paper, researchers Kenneth Ge and Andre Assis propose AgentArmor, a systematic framework to understand and mitigate these failures. They categorize failures into three mechanisms: underspecification (unsafe default behavior), capability errors (safe action available but not followed due to bias or limits), and agent harness errors (failure to execute safe action through the harness). The authors evaluate these across 8 distinct evaluations inspired by real-world deployment failures, covering 20 coding environments and 59 synthetic transcript templates.

AgentArmor introduces a set of practical mitigations designed as an agent harness modification. These include an extended system prompt to reinforce safe behavior, a separate command classifier to detect risky actions, a '3 strikes' policy that halts unsafe sequences, deterministic guardrails that block specific dangerous operations, and tools allowing the agent to edit its own context for self-correction. In statistically significant tests, AgentArmor demonstrated superior safety compared to baseline coding agents, reducing the incidence of harmful outputs across all three failure categories.

The research provides concrete recommendations for improving current coding agents and outlines a design philosophy for future agent harness features. By addressing failures at the system level rather than relying solely on model improvements, AgentArmor offers a scalable approach to making AI-assisted software development safer, especially as agents gain more autonomy in production environments.