Tree-based Credit Assignment for Multi-Agent Memory System

Researchers from the arXiv cs.MA community have proposed TreeMem, a novel credit assignment framework for multi-agent memory systems built on top of large language models. Current reinforcement learning approaches face a dilemma: they either apply the same final downstream reward (e.g., QA accuracy) to all agents—which is too coarse—or they design task-specific rewards for each subtask, requiring costly manual annotations like key evidence. TreeMem solves this by restructuring the typical builder–summarizer–retrieval pipeline into a tree. Each agent’s output branches into multiple subsequent paths, and the agent’s credit is estimated as the Monte Carlo average of rewards across those branches. This converts a single coarse final reward into agent-specific optimization signals, allowing heterogeneous agents (builder, summarizer, retriever) to be updated simultaneously and specialize more effectively.

Testing on long-horizon benchmarks, TreeMem consistently outperformed prior strong baselines, validating that tree-structured credit assignment can improve multi-agent coordination without expensive human labels. The approach is particularly valuable for real-world applications where long-context understanding is critical—such as document analysis, legal reasoning, or multi-turn customer support—and where manual annotation would be prohibitive. By using the inherent structure of the pipeline itself to derive per-agent rewards, TreeMem opens the door to more scalable and efficient training of multi-agent LLM systems.