SearchAuditor's new framework boosts AI search recovery to 32.3%

Deep search agents tackle complex questions through long, multi-step web interactions—but they're fragile. A small reasoning error can cascade through noisy trajectories into fluent, confident, and completely wrong answers. Diagnosing these failures means manually inspecting enormous execution traces, which quickly exceeds human capacity. To address this, researchers from Microsoft and academic labs introduced SearchAuditBench, a benchmark built to test whether LLM auditors can localize, attribute, and repair agent failures automatically.

SearchAuditBench contains 1,243 expert-annotated failed trajectories collected from eight open-weight models across five deep-search benchmarks. Each trajectory averages 73.1 messages and 65.1K tokens, with annotations marking the critical error step, a search-specific root cause, and a reference repair with grading rubrics. The team also proposed SearchAuditor, a multi-perspective auditing framework that uses evidence-grounded adjudication to identify where things went wrong and how to fix them. In experiments, even the strongest baseline powered by GPT-5.5 achieved only a 26.6% end-to-end pass rate—but SearchAuditor topped it with a 32.3% pass rate across frontier models. More importantly, resuming failed runs with SearchAuditor's repairs let agents recover from errors significantly better, pointing toward more robust autonomous search systems.