OLIVIA lets LLM agents learn from mistakes during deployment
New method adds a lightweight learning layer to ReAct agents, cutting errors without retraining.
Large language models powering autonomous agents (like ReAct) rely on reasoning-action loops, but small action-selection errors compound over multi-step tasks — leading to wasted tool calls, latency, and brittle behavior. Existing fixes rely on prompting or retrieval, which influence behavior indirectly and lack explicit decision layers for scoring actions or representing uncertainty. Enter OLIVIA, a new framework from researchers at the University of Illinois and Adobe that adds a lightweight online learning layer to ReAct-style agents. It treats the LLM's final action-selection step as a contextual linear bandit: candidate actions are scored using frozen hidden states as context, and feedback from each action is used to update the policy via UCB (upper confidence bound) exploration.
OLIVIA preserves the underlying reasoning process while providing trackable uncertainty estimates and sample-efficient online updates — all at minimal computational overhead. The team tested it on four benchmarks spanning multi-step decision tasks (web navigation, tool use, etc.) and found consistent improvements over static ReAct and prompt-based inference-time baselines. This suggests that explicit online decision layers offer a practical, scalable alternative to prompt engineering for agents that must improve during deployment. For enterprises running LLM agents in production, OLIVIA could mean fewer repeated API calls, lower latency, and more reliable task completion without costly retraining.
- OLIVIA treats LLM action selection as a contextual linear bandit, enabling online learning from action-level feedback during inference.
- Uses UCB (upper confidence bound) exploration to improve policy sample-efficiently with minimal computational overhead.
- Tested on 4 benchmarks, OLIVIA consistently outperformed static ReAct and prompt-based adaptation methods.
- Provides explicit uncertainty estimates for each candidate action, enabling trackable and fine-grained deployment-time adaptation.
- Preserves the underlying frozen LLM reasoning process — no retraining or model modification needed.
Why It Matters
OLIVIA makes deployed LLM agents smarter over time, reducing errors and costs without retraining.