Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes

A team of researchers from Concordia University and the University of Calgary has published the first comprehensive empirical study of failures in agentic AI systems. The paper, 'Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes,' analyzed 13,602 issues and pull requests from 40 major open-source agentic AI repositories like AutoGPT and LangChain. Using stratified sampling and grounded theory on 385 detailed faults, the researchers identified 37 distinct fault types grouped into 13 higher-level categories, along with 13 classes of observable symptoms and 12 categories of root causes.

Key findings reveal that agentic AI failures often originate from fundamental mismatches between probabilistically generated artifacts from LLMs and the deterministic constraints of external tool interfaces. Common failure patterns include dependency integration problems, data validation errors, and runtime environment handling issues. The researchers used Apriori-based association rule mining to identify statistically significant fault propagation pathways, such as token management faults leading to authentication failures and datetime handling defects causing scheduling anomalies.

The taxonomy was validated through a developer study with 145 practitioners, who rated it as highly representative of real-world failures (mean 3.97/5). Notably, 83.8% of participants reported that the taxonomy covered faults they had personally encountered in production systems. This research provides the first systematic framework for understanding, diagnosing, and preventing failures in increasingly complex AI agent architectures that combine LLM reasoning with external tool invocation.