Shared Selective Memory boosts LLM agent success to 96%, cuts tokens 97x

Agentic LLM systems that generate code through multi-turn tool use suffer from a fundamental context problem: each session starts from scratch, discarding valuable configuration choices, domain constraints, data schemas, and tool-use patterns. Naively persisting entire conversation histories is token-inefficient and actually degrades generation quality with stale traces. A new paper on arXiv introduces shared selective persistent memory, an architecture that identifies and retains only four categories of reusable context—task specifications, data schemas, tool configurations, and output constraints—while discarding session-specific reasoning traces. This memory is shared across users via workspaces with role-based access control, enabling collaborative reuse without redundant specification.

The system was deployed on a collaborative workspace platform where LLM agents produce, edit, and maintain git-versioned artifacts like dashboards, reports, and data-driven documents from heterogeneous sources (CSV, SQL, REST APIs, MCP servers). A complementary zero-token data refresh mechanism decouples generated programs from runtime data, allowing artifact reuse without re-invoking the LLM. Across three enterprise scenarios, shared selective persistent memory achieved 96% task completion (vs. 79% without memory and 71% with full history). Zero-token refresh eliminated LLM re-invocation for recurring updates, yielding a 14x task-time reduction, while summary-driven generation cut per-invocation token cost by 97x versus raw data injection. A replication on four public datasets confirmed generalizability, with zero-token refresh succeeding in all 12 trials.