DeepSeek V4 Flash fails rule-following tests locally

DeepSeek V4 Flash’s latest release is drawing criticism for its persistent inability to follow rule-based prompts, even in controlled local environments. Users report the model ignores explicit instructions regardless of language, formatting, or prompting techniques. This limitation undermines claims of frontier-level performance, as real-world applications often require strict adherence to rules or structured outputs.

The root issue appears to stem from DeepSeek V4’s aggressive context compression strategy. Rules, skills, and prompts are stored as compressed summaries—128 tokens squeezed into a single entry—across 43 neural network layers. Only 2 layers retain full detail, while 21 layers operate at a 4:1 compression ratio. Even with uncompressed recent tokens, rule-based instructions are often lost in translation, leading to inconsistent behavior. A workaround involving VLLM’s `--hf--overrides` argument may double effective context retention, but early tests suggest limited improvement.