DeepSeek V4-Flash hits 68 tok/s on 2× RTX 3090 with used Xeon server
Full 284B MoE checkpoint runs on 2020 GPUs with zero re-quantization — here's how.
A self-hoster ran DeepSeek V4-Flash-0731 — the full official 156 GB checkpoint, not a re-quant — on a used Dell R940 with 4× Xeon Platinum 8268, 768 GB DDR4, and 2× RTX 3090. They shared it because they couldn't find anyone else publishing Ampere results for this engine. The model is 284B total / 13B active MoE, with routed experts natively in MXFP4 and FP8 linears — nothing re-quantized. A custom vLLM fork (lvllmds4-x v2.3.8) runs a CPU-GPU hybrid MoE engine over system RAM, and a Marlin weight-only trick lets Blackwell-era checkpoints run on 2020 GPUs. Speculative decoding built into the checkpoint drives most of the single-stream speed. Used-server pricing works out to roughly $6K all-in for the full checkpoint.
- DeepSeek V4-Flash-0731 runs at 33 tok/s single-stream and 68 tok/s aggregate on 2× RTX 3090 + used quad-Xeon R940 (~$6K)
- Official 156GB checkpoint loaded natively — no re-quantization; MXFP4 experts and FP8 linears handled via Marlin on Ampere
- Custom vLLM fork (Lvllmds4-x v2.3.8) with lk_moe v2.3.1 hybrid engine enables CPU-GPU MoE on 4 NUMA nodes, using 170GB for one instance
Why It Matters
Proves huge MoE models can run on cheap used hardware, making local 284B-scale inference affordable for enthusiasts and small labs.