DeepSeek V4 tops coding leaderboards but lags 8 months behind frontier in general reasoning

DeepSeek V4 presents a striking paradox. Its Pro configuration achieves top-tier coding benchmark scores: 80.6% on SWE-bench Verified and 93.5 on LiveCodeBench. Yet when CAISI tested the same model across a wider range of domains, including cybersecurity and abstract reasoning, it concluded the model is roughly 8 months behind the US frontier—roughly the level of GPT-5. DeepSeek's own launch framing put it only 2 months behind the then-frontier. Both assessments are correct but measure different things. Coding leaderboards are heavily optimized and narrow in scope, so a high score there primarily reflects strong code generation, not general reasoning or agentic capabilities. CAISI's broader battery reveals gaps that leaderboards miss, especially as the frontier itself has advanced—note the unreleased closed model "Fable 5" that dropped this week.

The local deployment angle compounds the issue. The headline number often cited is the 1.6T parameter Pro config, which is impractical for most users. By the time you run a quantized Flash version that fits on a local machine, performance drops significantly from the headline benchmarks. For developers using local models for agent tasks—tool calling, planning, multi-step reasoning—the quantized DeepSeek V4 may not match the promise of its raw coding scores. Users report that once you factor in quantization and actual agentic workloads (not just code completion), the gap from the frontier feels larger than the 2-month claim. This discrepancy matters for professionals choosing a local model for production agent workflows.