llama.cpp b10236 adds DeepSeek V4 Lightning Indexer, boosting Apple Silicon speed 47%

The latest llama.cpp release, b10236, brings a major performance upgrade for Apple Silicon users running DeepSeek V4. The headline feature is a Metal implementation of the "DSv4 Lightning Indexer," a specialized kernel that optimizes attention over large KV caches. In benchmarks using a 512-token prompt and 128-token generation, prompt processing at a 30k cache distance improved 47%, from 33.40 to 49.18 tokens per second. Even at shorter distances, gains were notable—at 10k distance, speed rose from 73.90 to 86.95 t/s. Token generation also saw modest improvements, with tg128 climbing from 7.94 to 8.60 t/s at the same 30k distance.

Under the hood, the release stages and dequantizes K tiles in F16 threadgroup memory before simdgroup matrix loads, with zero-filled partial tiles and guarded stores ensuring consistent numerical paths across all KV segments. The Lightning Indexer supports 128-dimensional, 64-head inputs with F32 queries and weights, plus F16 keys and masks, and works with F32, F16, BF16, Q4_0, Q4_1, Q5_0, Q5_1, and Q8_0 K caches. Notably, the commits were "Assisted-by: Codex," showing AI-assisted development now in production LLM tooling. For developers running DeepSeek V4 on macOS or iOS, this update meaningfully reduces long-context prompt latency, making local inference more practical.