ParamMem: Augmenting Language Agents with Parametric Reflective Memory

A research team led by Tianjun Yao has published a paper introducing ParamMem, a novel parametric memory module designed to enhance the reasoning capabilities of language agents (AI systems that can take actions). The core innovation addresses a critical limitation in current agent architectures: repetitive outputs during self-reflection cycles that hinder performance. By analyzing reflection patterns across multiple task attempts, the researchers identified a strong positive correlation between reflective diversity and task success, motivating their new approach. ParamMem encodes these cross-sample reflection patterns directly into the model's parameters, creating a persistent memory of successful reasoning strategies that can be accessed and varied.

Building on this module, the team proposes ParamAgent, a comprehensive agent framework that integrates ParamMem with traditional episodic memory. This combination allows agents to generate more diverse and effective reflections through temperature-controlled sampling, essentially varying the 'creativity' of their self-critique. Extensive testing across code generation, mathematical reasoning, and multi-hop question answering demonstrated consistent performance gains over state-of-the-art baselines. Crucially, the system is sample-efficient and enables 'weak-to-strong' knowledge transfer, meaning a smaller model can improve by learning from the reflection patterns of a larger one. This paves the way for more autonomous AI systems capable of genuine self-improvement without relying on constant feedback from more powerful external models like GPT-4 or Claude 3.5.