ECHO: New method teaches AI agents to adapt turn-by-turn using epistemic credit
Belief-agnostic policies compound errors exponentially—ECHO fixes that with per-turn Bayesian rewards.
Deep Dive
ECHO (Epistemic Credit for History-Conditioned Optimization) is a policy-gradient algorithm that assigns turn-level credit using posterior-sensitive rewards in multi-turn language agents, built on a new Epistemic Decision Process (EDP) framework. On the Clue Selector Game, ECHO outperforms trajectory-level GRPO, improving resolution, information gain, and efficiency while producing almost no visible reasoning text.
Key Points
- Introduces Epistemic Decision Processes (EDPs) to model multi-turn information seeking with explicit belief states.
- ECHO uses a clipped policy-gradient objective with per-turn Bayesian rewards, outperforming trajectory-level GRPO on the Clue Selector Game.
- Achieves strong epistemic metrics (grounding, recovery, calibration) with minimal visible reasoning—gains in efficiency and resolution.
Why It Matters
Makes AI agents smarter about when to think, cutting wasted compute and compounding errors in multi-turn tasks.