XAI for Coding Agent Failures: Transforming Raw Execution Traces into Actionable Insights

A new research paper by Arun Joshi tackles a critical pain point in AI-assisted software development: the opaque failures of LLM-based coding agents. While models like GPT-4 can generate code, their execution traces when they fail are notoriously difficult for developers to interpret. Joshi's systematic explainable AI (XAI) approach introduces a three-component framework designed to bring clarity to this chaos. It first establishes a domain-specific failure taxonomy built from analyzing real agent failures, then uses an automatic annotation system to classify errors, and finally employs a hybrid generator to produce visual execution flows, natural language explanations, and actionable recommendations.

The impact is quantified through a user study with 20 participants, split between technical and non-technical backgrounds. The results are significant: developers using this structured XAI system identified the root causes of coding agent failures 2.8 times faster than when wrestling with raw execution traces. More importantly, they were able to propose correct fixes with 73% higher accuracy. The research demonstrates that this methodical, domain-aware approach provides more consistent and useful insights than asking a general-purpose LLM for an ad-hoc explanation, effectively bridging the interpretability gap between powerful AI agents and the human developers who rely on them.