AI That Remembers: When It Helps, When It's a Waste

AI agents are becoming digital assistants that can take real actions — booking flights, managing files, or updating spreadsheets. For them to work well, they often need to remember what happened before. A new study tested exactly when that memory is worth the money. The researchers created a benchmark called MERIT, which gave AI agents a series of tool-use tasks and measured how much their performance improved when they could recall earlier facts.

The results were striking. On tasks that genuinely depended on remembering something from an earlier episode, memory lifted success rates from 0% all the way to 55–100% across models. But on tasks that didn't require memory, adding it was pure expense. The team also found that swapping one type of memory for another could change task success by up to 60 percentage points — meaning the underlying design of the memory matters as much as whether it exists at all.

Cost is the other big finding. Replaying the full history of past interactions — a common approach — was never economical. The best memory setups delivered only 2.7 to 3.9 times their marginal usefulness per dollar spent. In other words, even when memory helps, you're often paying far more than you should for a modest boost.

Perhaps most surprising: memory alone isn't enough. When an AI correctly retrieved a fact it needed, it then acted on that fact only 55% of the time. It can remember the right answer and still ignore it. The takeaway for users: memory is not a magic upgrade. It helps only on the right tasks, at the right cost, with the right design — and even then, the AI may not put it to use.