Huawei's new framework turns AI agent trajectories into verified evidence

A team of 16 Huawei Noah's Ark Lab researchers, including Ruiming Tang and Kun Gai, published arXiv:2608.05235 introducing a framework for converting AI research-agent trajectories into auditable experimental records. The core problem: when agents run multi-round machine-learning experiments in industrial recommendation systems, their logged trajectories are not automatically trustworthy. Artifacts can be unsupported or incomplete, executed rounds can be invalid or confounded, and later modifications can obscure earlier findings. The paper formalizes this as 'trajectory-to-evidence conversion' and asks what a completed research process has actually established.

The solution is an evidence-grounded pipeline combining a context-isolated generate–verify–repair process, which checks artifacts for evidence violations and missing downstream requirements before release, with post-execution claim qualification. After execution, validity and attribution checks consolidate evidence across rounds, qualifying intervention-level claims as actionable repairs, diagnostic guards, or withheld findings. A hybrid LLM-assisted controller then applies, defers, or rejects records based on available target evidence. Record audits show that affirmative applicability judgment is a bottleneck for the controller. In paper-to-target adaptations, later rounds often improve on the first, but final rounds frequently underperform an earlier best—exposing non-monotonic trajectory evolution. Most importantly, candidates produced through the complete workflow yielded positive online lifts relative to deployed baselines, demonstrating real-world impact beyond academic validation.