AdaMAST's adaptive failure taxonomy boosts AI agents by up to 15% on coding benchmarks

A team of researchers (Mert Cemri, Andrei Cojocaru, Matei Zaharia, Ion Stoica, and others) has published 'Fantastic Adaptive Taxonomies and How to Use Them' on arXiv, presenting AdaMAST (Adaptive MAST). The core idea: instead of relying on raw, instance-specific execution traces as feedback for improving agent systems, AdaMAST automatically induces a structured failure taxonomy from those traces. The taxonomy consists of named failure codes organized along three fixed axes—system-level, role-specific, and domain-specific—with every name, definition, and evidence pattern derived without any hand-authoring or human annotation. The result is a compact (order-of-magnitude compression) and human-faithful representation that matches expert annotations more closely than a hand-crafted reference vocabulary.

AdaMAST's taxonomy serves as a shared feedback interface across three improvement procedures. In agent-system search, taxonomy-coded diagnoses outperform free-form reflection on all five tested benchmarks. For runtime monitoring, taxonomy feedback improved SWE-agent's resolution on SWE-bench Verified Mini from 60% to 70%, and Claude Code from 64.0% to 70.7% as a runtime skill. In trajectory selection, the AdaMAST-Judge verifier improved best-of-5 accuracy on Terminal-Bench 2.0 by 8–15 points over Pass@1. The induced vocabularies are adaptive—taxonomies for different domains share few codes—making them reusable wherever failure feedback is needed, without altering model weights.