DeepSeek-V4-Flash hits 17.2 t/s on A6000 with 1M context

A Reddit user shared detailed performance benchmarks for the DeepSeek-V4-Flash-0731 UD-Q8_K_XL quantized model running entirely on a high-end workstation. The setup pairs an AMD EPYC 74F3 24-core CPU with 256GB of 8-channel DDR4-3200 RAM and an NVIDIA RTX A6000 48GB GPU. Out of the box, the user measured inference at a steady 17.20 tokens per second, with prompt processing sitting in the high 70s t/s before degrading to mid-30s t/s at 300k context. Crucially, the 48GB VRAM was enough to load the full 1 million token context window, though prompt processing at that scale would be impractical due to severe slowdowns.

The post gained traction after the user discovered a simple configuration tweak. By increasing the batch size parameter to 8096, prompt processing jumped to almost 400 tokens per second at the start, settling around 300 t/s at 20k context. This represents a 4-5x improvement over the stock 70 t/s, highlighting how batch size optimization can dramatically impact throughput for long-context models. The user compared their results to Apple M5 Macs, noting their setup is less flashy but delivers competitive numbers. The benchmark is particularly relevant for professionals running local LLMs with massive context windows, as it demonstrates that a single RTX A6000 can handle both large models and extreme context lengths with the right tuning.