Zero-Mem kills LLM memory tokens, cutting agent memory costs 57.6%

LLM agents depend on memory to stay consistent over long interactions, but traditional memory systems burn extra tokens and time by calling LLMs to write and read intermediate records. Zero-Mem, from Yilin Xiao and 10 co-authors, inverts this design: it never invokes an LLM during memory operations. Instead, it keeps the original interaction traces as the single source of truth and organizes them into two complementary structures—an entity-context graph that exposes cross-interaction connections, and a temporal hierarchy that preserves conversational locality and session state. For each query, Zero-Mem weighs both views, retrieves from each, and follows their structure to recover supporting relations or surrounding context. A deterministic calibration step discards conflicting evidence before the final-QA reader, the only LLM call, grounds its answer in the retrieved traces.

Across long-memory and long-context question-answering benchmarks, Zero-Mem achieves competitive accuracy while eliminating all LLM calls and token consumption from memory operations. With the same final-QA reader and context budget, it reduces memory-operation time cost by 57.6% relative to the fastest compared baseline. Ablations confirm that both the graph and temporal views contribute, and that their query-dependent coordination is key. The results challenge the assumption that structured agent memory must generate an intermediate representation of the past. The authors plan to release code and implementation details after peer review, which could let developers build cheaper, faster agents with memory that scales without token bloat.