MEMORA gives robots episodic memory, improving planning by 16.6%

Robots have long struggled with long-horizon planning because they lack the kind of episodic memory humans use to recall past experiences. A new paper from researchers introduces MEMORA (Embodied Action Memory), a system that gives robots the ability to form, maintain, and use persistent memory from egocentric video. MEMORA organizes memory into four typed stores: Environment Memory (places), Entity Memory (objects with state histories), Activity Memory (procedures), and Inferred Knowledge (regularities from repeated actions). It operates through a formation-consolidation-retrieval lifecycle: online editing updates object identities and state as new observations arrive, while offline consolidation abstracts repeated experiences into reusable procedures.

To test MEMORA, the team created MEMORA-Bench using 45 hours of extended EPIC-KITCHENS-100 video across 18 participants. They evaluated memory-grounded planning against four open-weight language models. The full MEMORA system—combining editable typed stores and consolidation—achieved the strongest results: up to 20.5 points improvement in memory-assessment accuracy and up to 16.6% relative improvement in out-of-distribution robot-grounded plan scores over the best baseline. A qualitative robot deployment study showed how memory-grounded language plans can interface with downstream control. The work was presented as an oral at the RSS 2026 workshop on Foundation Models for Robot Planning.