MILES framework boosts LLM reasoning with modular memory and learnable selection
New method lets LLMs accumulate and apply past reasoning steps to improve future answers.
Large language models (LLMs) increasingly improve reasoning through additional computation at test time, but most treat each problem in isolation. Existing memory-based approaches either store rigid whole-solution templates that fail on novel problems or use heuristic step selection unoptimized for final correctness. Learning-based selection policies require extensive training data and fixed action spaces, making them impractical for test-time settings where memory grows incrementally with limited supervision.
MILES solves this with modular memory units—asymmetric pairs of sub-goal embeddings and sub-instructions, each linked to a learnable selection head. A coarse-to-fine retrieval mechanism first enables memory expansion and supervised training of selection heads from confident samples, then applies learned heads to rerank candidates and guide reasoning for uncertain cases. Experiments show MILES consistently matches or outperforms prior methods while delivering better accuracy-efficiency tradeoffs, robustness, and task transferability.
- Modular memory uses asymmetric pairs of sub-goal embeddings and sub-instructions with learnable selection heads
- Two-stage coarse-to-fine retrieval: memory expansion with supervision from confident samples, then reranking for uncertain ones
- Matches or surpasses existing memory-based reasoning methods with superior accuracy-efficiency tradeoffs
Why It Matters
Enables LLMs to continuously improve reasoning from accumulated experience without costly retraining or large datasets.