AI Video Tools May Only Pretend to Remember What They See

Video AI models — the kind that track objects, people, or scenes across a long clip — are built with a 'memory,' a store of earlier frames they can consult. Until now, everyone assumed the benefit came from pulling up the right past content. This team tested that assumption directly by swapping the retrieved memory for something else at the exact moment the model used it. That trick, called read-time substitution, separates 'memory helps' from 'the specific memory helps.'

The results were surprisingly split. In several video 'world models,' replacing real memory with content-free filler recovered essentially the full benefit — 102% on one dataset and about 70% on another. In other words, those models weren't really using what they remembered; they just performed better with some memory-shaped support in place. Meta's SAM 2 behaved the opposite way. Handed a valid but wrong memory on a standard video dataset, its accuracy on picking out objects crashed from 0.926 to 0.182. On a task where something reappears after being hidden, it fell from 0.459 to zero.

Why does this matter outside a lab? These systems sit inside video editing tools, security cameras, robot navigation, and self-driving perception. A model that only pretends to remember fails at exactly the moments memory is supposed to save it: when a person walks behind a car and steps back out, or when an item is briefly occluded. If you're relying on AI to track something important over time, 'it has memory' is not the same as 'it remembers correctly.'

The catch: this study examined specific frozen models and datasets, so it isn't proof that all video AI is faking it. But it hands developers a cheap, direct test for checking whether a memory claim is real — and gives buyers a good question to ask before trusting a system with anything that matters.