AI Assistants Forget What They Did — New Research Shows How to Fix It

AI helpers are moving from chatting to doing — booking flights, filling forms, chasing invoices. Each step leaves a record: what the AI tried, what tool it used, what came back. After a few hundred steps, that history is far too long to hold in the AI's working memory (its "context window," the limited amount of text it can keep in mind at once). So the AI has to look things up later, a bit like searching your own old notes.

The problem, this paper argues, is how that searching gets done. Today's systems chop the history into fixed-size blocks and reward finding any single relevant snippet. But real answers often need three or four steps stitched together, and scattered snippets don't add up to a complete picture. The researchers tested a different approach on 2,000 question sets drawn from 1,207 recorded runs of a real AI agent. Instead of blocks, they built "provenance units" — records tied directly to the actual tool call and its output, so any hit points back to the exact original step. Then they added a connection map, showing which steps led to which others, to boost scores for related steps.

The results were solid. Getting the complete evidence inside a limited 2,048-slot memory budget improved by 19.07 points over fixed 512-word blocks, and stayed 11.96 points ahead of the best possible block size chosen after the fact. The connection map added a further 4.55 points — modest, but concentrated exactly where it helps most: when the needed evidence is spread across several separate steps. Entity lists and other shortcuts didn't produce the same benefit, suggesting the links between steps are doing real work.

One honest catch: this is a research paper, not a product. Nothing ships tomorrow, and lab results don't always survive contact with messy real-world tasks. But the direction matters. As AI agents take on longer chores — expense reports, research, multi-day bookings — remembering accurately becomes the bottleneck. Expect the assistants you already use to quietly get better at not asking you the same thing twice.