DeepSeek V4 Flash runs 32 tok/s on single AMD Ryzen AI MAX+ 395 with 128 GB unified memory
Record-breaking local inference: 284B model at 32 tok/s on a laptop chip.
DeepSeek V4 Flash, a 284-billion-parameter mixture-of-experts model, now runs locally on AMD's top-end laptop chip. The team behind the work—shared in a blog post on Lucebox—managed to fit the full model plus its speculative draft onto a single Ryzen AI MAX+ 395 with 128 GB of unified LPDDR5X memory. The result: a decode rate of up to 32 tokens per second at q=4 verification. Without speculative decoding, the autoregressive rate reaches 25.3 tok/s. The run was submitted to the LocalMaxxing leaderboard, where it sits 68.5% ahead of the next fastest entry for the Radeon 8060S and 2.05× faster than the previous leader in the Ryzen AI Max 395 group.
The breakthrough relies on a custom quantization family called ROCmFPX, which crams the model into 102.3 GB—roughly 2.88 bits per parameter. The technique uses mixed precision: ROCmFP2 for routed-expert gates, ROCmFP3 for expert down projections, and ROCmFP4 for denser layers. The speculative decoder, dubbed DSpark, proposes up to three tokens in a compact draft pass, then verifies four positions simultaneously through a fused HIP graph. The code is open-source under Apache-2.0 and ready for anyone with the target hardware to reproduce the results. This marks a major step for running frontier-level models entirely on consumer-grade hardware.
- DeepSeek V4 Flash achieves 32 tok/s on a single AMD Ryzen AI MAX+ 395 with 128 GB unified memory.
- Uses ROCmFPX quantization at ~2.88 bits per parameter to fit the 284B model into 102.3 GB.
- Speculative decoding (DSpark) adds 26.4% performance over the 25.3 tok/s autoregressive rate.
Why It Matters
Bringing 284B parameter models to consumer hardware enables powerful local AI without cloud dependencies.