Strands Evals' Detectors pinpoint AI agent failures and root causes in minutes

Traditional AI agent evaluations answer "how well did the agent do?" but leave teams manually reviewing execution traces to understand why a 60% goal success rate dropped. For teams operating agents at scale, this manual bottleneck turns detection into a multi-hour debugging session. Strands' Detectors, part of the Strands Evals SDK, automate that workflow. They analyze every span in a session against a comprehensive failure taxonomy covering hallucination, incorrect actions, orchestration errors, task instruction non-compliance, execution errors, context handling errors, repetitive behavior, LLM output issues, and configuration mismatch. For each failure, Detectors return the span location, one or more categories, a confidence score, and evidence extracted from the trace.

The pipeline runs in two phases: failure detection scans each span, then root cause analysis traces causal chains between failures—separating primary root causes from downstream symptoms. It classifies each failure as PRIMARY, SECONDARY, or TERTIARY, determines propagation impact, and generates fix recommendations that specify whether the change belongs in the system prompt, tool descriptions, or elsewhere. The system handles sessions of any size via a tiered strategy: direct analysis for small traces, failure path pruning for medium sessions, and chunked analysis for very large ones. Teams can integrate detectors into their evaluation pipelines for automatic diagnosis on every test run, eliminating the senior-engineer bottleneck and turning hours of trace inspection into structured, automated output.