MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games

A research team from multiple institutions has introduced MEMO (Memory-augmented Model Context Optimization), a novel framework designed to solve a critical problem in multi-agent AI systems: unreliable performance in complex, multi-turn games. Current evaluations of LLM-based agents in scenarios like negotiations or strategic games suffer from high variance, where small early deviations compound across turns, making win rates and rankings unstable across repeated runs. MEMO addresses this through a two-pronged self-play approach that optimizes the context provided to models during inference.

The framework combines 'retention'—maintaining a persistent memory bank of structured insights from previous game trajectories—with 'exploration' that runs tournament-style prompt evolution. It uses TrueSkill for uncertainty-aware selection and prioritized replay to revisit rare but decisive game states. In testing across five text-based games, MEMO demonstrated remarkable improvements, nearly doubling win rates for both OpenAI's GPT-4o-mini (from 25.1% to 49.5%) and Alibaba's Qwen-2.5-7B-Instruct (from 20.9% to 44.3%) after 2,000 self-play games per task.

Beyond raw performance gains, MEMO significantly reduces run-to-run variance, providing more stable and reliable rankings across different prompt variations. The paper notes the largest improvements occur in negotiation and imperfect-information games, while traditional reinforcement learning remains more effective in perfect-information settings. This research, published on arXiv, suggests substantial untapped potential for improving multi-agent LLM robustness through sophisticated context optimization techniques rather than just model scaling.