CALMem gives LLMs unbounded memory without model changes

CALMem (Conversational Application-Layer Memory) is a new dual memory architecture designed to solve the fundamental limitation of fixed context windows in large language models (LLMs). Developed by researchers Rajendra Narayan Jena, Rajan Padmanabhan, and Sankar Arumugam, the system operates entirely at the application layer, meaning it requires no modifications to the underlying LLM and works with any provider. The architecture consists of two complementary memory subsystems: an episodic memory layer built on sliding-window vector embeddings that captures conversation history, and a semantic memory layer that stores structured facts written by the AI agent itself. A central component is the MOIM (Message of Injected Memory), a token-budget-adaptive injection mechanism that automatically retrieves and injects relevant past context at each turn, scaling injection depth inversely with the current context pressure.

A standout contribution is CALMem's support for intra-session retrieval—compacted-away turns from the current session remain searchable, a gap that prior work like MemGPT did not address. This means even after the context window is full and older turns are compressed, the system can still recall information from earlier in the same conversation. Implemented in a production Rust codebase, CALMem is provider-agnostic and degrades seamlessly to the original LLM behavior with zero overhead when disabled. The paper details the architecture, design decisions, and performance trade-offs, positioning CALMem as a practical, model-independent solution for building conversational AI systems that maintain coherent, long-running dialogues.