LANTERN recovers 78% of lost LLM context with zero inference cost

Large language models regularly forget critical details when conversation histories are compressed to fit within finite context windows. A new paper from researcher Rahul Subramani introduces LANTERN (Layered Archival and Temporal Episodic Retrieval Network), a lightweight memory layer that proactively archives every conversation turn and restores relevant details after compaction using hybrid retrieval—all without making a single LLM call. The system adds fewer than 25 milliseconds of latency per turn, making it practical for real-time applications.

In rigorous testing across 94 real multi-turn conversations containing 1,894 human-validated ground-truth facts (with high inter-rater reliability, kappa=0.81), LANTERN-Rerank recovered 78.3% of verifiable facts lost to compaction. This significantly outperformed a faithful reimplementation of MemGPT's LLM-driven extraction and multi-query search pipeline (72.4%; Wilcoxon p<0.0001, 95% CI [+3.1, +8.6] pp, d=0.43)—at a fraction of the inference cost. Even without the reranker, base LANTERN matched or exceeded the LLM-driven baseline (p=0.005) using zero LLM calls. When four production LLMs answered fact-bearing questions using LANTERN-restored context, accuracy improved by an average of 8.4 percentage points (Wilcoxon p<0.05 for each model individually), demonstrating the recovered context is useful across diverse model architectures. The author has released the full evaluation framework to support reproducibility.