Llama.cpp adds DeepSeek V4 Flash 0731 support
DeepSeek V4 Flash 0731 now runs locally via llama.cpp with structured output and reasoning control
The open-source Llama.cpp project by ggml-org has added native support for DeepSeek's V4 Flash 0731 model, expanding local AI inference capabilities. This update introduces new chat templates aligned with DeepSeek's official encoders, enabling structured output generation and fine-grained reasoning control. Developers can now utilize `drop_thinking` to omit prior reasoning steps or preserve them when tools are active, mimicking the model's original behavior.
The integration includes platform-specific optimizations across 14 architectures, from Apple Silicon to NVIDIA CUDA 13.3 and AMD ROCm. It also addresses parser fixes for tool call separators and empty tool arguments, ensuring compatibility with DeepSeek's latest updates. The update is particularly valuable for edge deployments, where structured outputs and controlled reasoning reduce cloud dependency costs.
- New DeepSeek V4 Flash 0731 templates in Llama.cpp support structured output and reasoning control (`drop_thinking`, `preserve_reasoning`)
- Multi-platform support: CUDA 12/13, Vulkan, OpenVINO, and 12+ OS/arch combinations (Apple Silicon, Android, Windows, Linux)
- Fixes parser issues for tool calls and empty arguments; aligns with DeepSeek's official encoding specs
Why It Matters
Enables local deployment of DeepSeek V4 Flash 0731 with enterprise-grade features, cutting cloud costs and improving latency for AI apps.