From Local Corrections to Generalized Skills: Improving Neuro-Symbolic Policies with MEMO
A new AI framework turns specific verbal feedback into general, reusable skills for robots.
A research team led by Benjamin A. Christie has introduced MEMO (Memory Enhanced Manipulation), a novel framework designed to overcome a fundamental bottleneck in neuro-symbolic robot policies. These policies use large vision and language models to break tasks into semantic subtasks but are constrained by a fixed library of pre-defined skills (like motion primitives). When a robot lacks the right skill, it fails. MEMO addresses this by leveraging intuitive human feedback—like saying "no, go higher"—to dynamically create new, generalized skills, moving beyond simple recall of specific corrections.
The system works by collecting, clustering, and re-phrasing natural language corrections across multiple users and tasks. It synthesizes this aggregated feedback into general text guidance and coded skill templates, storing them in a retrieval-augmented skillbook. At runtime, the robot's policy can retrieve relevant text and code from this memory, enabling it to generate new skills on the fly and reason over multi-task human feedback. Experiments show MEMO successfully aggregates local corrections into generalizable skill templates, allowing robots to tackle novel tasks where existing baselines fall short, marking a significant step toward more adaptable and teachable robotic systems.
- MEMO uses a retrieval-augmented skillbook to store synthesized skills from human feedback.
- It clusters and generalizes specific verbal corrections (e.g., "go higher") into reusable code and text templates.
- The system enables neuro-symbolic policies to tackle novel tasks by dynamically expanding their skill library.
Why It Matters
It enables robots to learn continuously from everyday human interaction, moving beyond rigid, pre-programmed capabilities.