Procedural Memory Distillation boosts LLM self-improvement by up to 13.6%
New method captures cross-episode signals to make models learn from past failures.
Deep Dive
Procedural Memory Distillation (PMD) converts cross-episode rollout signals into reusable procedural memory and distills it into the model's weights during training. On Qwen3-8B and OLMo3-Instruct-7B, PMD improves over SDPO by 3.8–5.5% on SCIKNOWEVAL and 7.9–13.6% on LIVECODEBENCH. The co-evolution of policy and memory is key—freezing either component causes more than a 10% performance drop across SCIKNOWEVAL domains.
Key Points
- PMD captures cross-episode rollout signals (strategies, failures, patterns) and distills them into model weights.
- Outperforms SDPO by 3.8–5.5% on SCIKNOWEVAL and 7.9–13.6% on LIVECODEBENCH using Qwen3-8B and OLMo3-Instruct-7B.
- Co-evolution of policy and memory is critical—freezing either component causes >10% performance drop.
Why It Matters
Enables LLMs to autonomously learn from experience, reducing need for manual fine-tuning and human feedback.