Why Your LLM Agent Keeps Forgetting — and the Submodular Fix That Changes Everything
Recency truncation is broken—new algorithm picks relevant context at query time.
LLM agents accumulate context from multiple sources—user turns, persistent memory, and tool outputs (file reads, API responses). As sessions grow, context exceeds token budgets. The standard fix, recency truncation, is topic-blind: it discards old facts even when the current query asks about them, while retaining verbose but irrelevant recent content. This fails for agents needing long-term recall. Existing alternatives like RAG or compression are either external or query-blind.
PACMS solves this by treating all context as a single pool and applying submodular optimization to select the most relevant items given the current query. It is pluggable, operating at prompt assembly time inside the agent framework. This ensures that memory entries, conversation history, and tool outputs are selected by relevance, not recency. The approach is query-aware and lossless in selection—only irrelevant items are dropped, not summarized. Early results suggest significant improvements in factual recall across long agent sessions.
- Recency truncation discards early but relevant facts, even when the current query targets them.
- PACMS treats memory, conversation turns, and tool outputs as a single candidate pool for selection.
- Uses submodular optimization for query-aware, lossless context selection at prompt assembly time.
Why It Matters
Enables LLM agents to reliably recall information across long sessions without losing critical early context.