ReflectWorld-MM: AI memory system tracks entities across endless video streams

ReflectWorld-MM, described in a newly released ArXiv paper by Xiaokang Ma, Yifan Sun, and colleagues, tackles a fundamental challenge in AI: giving agents long-term memory for continuous video streams. Existing systems either cram everything into a model's context window or store flat feature vectors indexed by frames, making it hard to track recurring entities like people or objects across time. ReflectWorld-MM rethinks that design entirely: it extracts entity-resolved observations via a perception front-end with a bounded short-term memory, then stores them in a hierarchical long-term memory system grounded in human memory theory. This hierarchical structure consists of a multi-scale episodic memory for specific events, an evolving entity-centric semantic memory that accumulates knowledge about each entity, and a procedural memory for learned behaviors.

The system is built for real-world deployment: it can ingest arbitrary video streams and integrate with existing AI assistants without modification. Across six rigorous benchmarks spanning long-video understanding and lifelong memory tracking, ReflectWorld-MM achieved top accuracy on every single one, outperforming strong baseline memory agents as well as frontier models that lack dedicated persistent memory. This suggests that organizing memory around entities—rather than frames or raw timestamps—significantly improves an agent's ability to reason over accumulated visual experience. For applications like surveillance, personal assistants, and autonomous systems that need to operate continuously, ReflectWorld-MM points a clear path forward.