MemoryVAM gives robot video models episodic memory, boosting success 8.5x
New memory mechanism lifts robot task success from 5% to 42.5% on long-horizon benchmarks.
MemoryVAM integrates an episodic memory mechanism into video-world-model policies for robot manipulation. It uses a Recap-Cue module with a Perceiver-based compressor that maps CLIP embeddings into compact memory tokens, and a Cue Gate that estimates task completion from memory and language. The memory tokens are injected into both the video backbone and action decoder. On LIBERO-Mem, success improves from 5% to 42.5%. Real robot tests achieve 78.3% on counting tasks, 80.0% on spatial recall, and 75.0% on sequential tracking. The same mechanism works with UNet and Diffusion Transformer backbones.
- MemoryVAM adds episodic memory to video-world-model policies using a Recap-Cue module (Perceiver compressor + Cue Gate).
- On LIBERO-Mem, success rate jumps from 5% to 42.5% — an 8.5x improvement.
- Real robot tasks achieve 78.3% (counting), 80% (spatial recall), and 75% (sequential tracking).
Why It Matters
Enables robots to remember past actions for complex, long-horizon manipulation tasks without explicit progress labels.