DeepSeek's FlashMemory-V4 shrinks KV cache by 86.5% with LSA
New paradigm predicts future token needs, slashing memory use to 13.5%.
Conventional LLMs load full KV cache during decoding, creating a severe GPU memory bottleneck for ultra-long contexts. DeepSeek's FlashMemory-DeepSeek-V4 (FM-DS-V4) tackles this with Lookahead Sparse Attention (LSA), a novel inference paradigm that proactively predicts which historical tokens will be needed next. Instead of attending to all tokens, LSA uses a Neural Memory Indexer—a lightweight dual-encoder trained independently via standard retrieval frameworks without ever loading the massive backbone model. This decoupled training strategy allows the indexer to learn which KV chunks are query-critical, keeping only those in GPU memory.
Benchmark results across LongBench-v2, LongMemEval, and RULER show FM-DS-V4 compresses the physical KV cache footprint to just 13.5% of the full-context baseline, while maintaining or slightly elevating downstream accuracy (+0.6% absolute margin on average). At extreme 500K-token scales, memory overhead drops by over 90% without destabilizing core reasoning. The system also acts as an effective attention denoiser for tasks relying on long-term global memory. DeepSeek has released the paper, code, and model weights on GitHub and HuggingFace.
- FlashMemory-DeepSeek-V4 uses Lookahead Sparse Attention (LSA) to predict future token needs, reducing KV cache to 13.5% of baseline.
- Neural Memory Indexer is trained independently as a dual-encoder, never loading the backbone LLM into GPU memory.
- At 500K-token context, physical cache overhead drops over 90% while accuracy improves by +0.6% on average across long-context benchmarks.
Why It Matters
Enables ultra-long-context LLM serving on existing GPUs, dramatically cutting costs and latency for enterprise document analysis.