Experiential Reflective Learning for Self-Improving LLM Agents
New method lets AI agents learn from past mistakes, improving task success rates without retraining.
A team of researchers has introduced Experiential Reflective Learning (ERL), a novel framework designed to make large language model (LLM) agents smarter over time by learning from their own experiences. Unlike current agents that treat every new task from scratch, ERL enables agents to reflect on their past successes and failures. After completing a task, the system analyzes the entire interaction trajectory and outcome to generate concise, actionable "heuristics"—essentially lessons learned. These heuristics are stored in a knowledge base for future use.
When the agent faces a new task, ERL retrieves the most relevant past heuristics based on the current context and injects them into the agent's prompt. This provides guided, experience-based reasoning without requiring costly model retraining. The researchers tested ERL on the challenging GAIA2 benchmark, a test of general AI assistants on real-world web tasks. The framework achieved a 7.8% higher success rate compared to the strong ReAct baseline and outperformed other experiential learning methods. Key findings from their analysis show that selective retrieval of heuristics is crucial, and that these distilled lessons offer more transferable knowledge than simply showing the agent a few past examples.
- ERL improved agent success rate by 7.8% on the GAIA2 benchmark over a ReAct baseline.
- The framework generates reusable "heuristics" from single task attempts, enabling knowledge transfer across problems.
- Selective retrieval of relevant heuristics was found to be more effective than few-shot example prompting.
Why It Matters
This moves AI agents from static tools to adaptive systems that improve with use, crucial for reliable real-world automation.