Ollama v0.30.8 improves prompt caching and MLX stability
New release fixes provider selection and boosts KV cache reuse for faster inference.
Ollama's latest release, v0.30.8, addresses a key bug where the launcher could select the wrong GPU provider in certain system configurations, ensuring users get the correct backend for their hardware. More importantly, the update reworks prompt caching logic: by decoupling caching from context shift, the system now reuses the KV cache more aggressively during multi-turn conversations, reducing latency and memory overhead for long chat histories.
On Apple Silicon, the MLX inference path receives several stability enhancements. Linear and embedding layers now use hardened computation, and the MLX runner creates snapshots during both prompt processing and speculative decoding — making it easier to recover from errors or pause/resume inference. Recurrent model support also gets a boost with per-boundary states from gated-delta kernels, improving performance and accuracy for RNN-style architectures like Mamba or RWKV.
- Fixed bug where `ollama launch` could select the wrong provider in certain setups
- Prompt caching now decoupled from context shift, enabling better KV cache reuse for faster multi-turn responses
- MLX inference hardened with snapshot creation; recurrent models improved via per-boundary states from gated-delta kernels
Why It Matters
Local LLM runners get more reliable and faster; Apple users benefit from stable MLX inference.