Ollama boosts AI prefill speed by up to 7.9%

Ollama has released v0.32.10-rc0, a performance-focused update that significantly speeds up the prefill phase of AI models—particularly those using nvfp4 (NVIDIA FP4) quantization. The optimization targets the computational overhead of applying a global float32 scale to projection outputs, which previously required separate kernel launches and intermediate memory allocations. By merging the multiply and cast operations into a single kernel, Ollama reduces kernel launch latency and eliminates unnecessary materializations, leading to measurable speed improvements.

Benchmark results on an M5 Max (using medians from order-swapped A/B tests) show a 7.9% increase in prefill throughput for the qwen3.6:27b model (703 → 769 tokens/sec) and a 6.7% increase for muse-glimmer:30b (790 → 843 tokens/sec). Notably, speculative decoding performance remained unchanged, and the optimization only affects checkpoints with a global scale, leaving single-scale nvfp4, mxfp8, and affine checkpoints unaffected. This makes the update a targeted performance boost for quantized models relying on global scaling.