Ollama v0.32.4 pre-release boosts quantization quality and MLX support
Ollama's latest pre-release improves quantization for lm_head and fixes Qwen 3.5 experts.
Ollama announced v0.32.4-rc0, a pre-release that addresses several quantization-related issues and expands hardware support. The update, contributed by jessegross, dhiltgen, and ParthSareen, includes 8 commits since v0.32.3. Key technical changes: lm_head tensors are now quantized at 8-bit within the requested family, improving memory efficiency for models like Llama and Mistral. Additionally, quantization of a draft model's output head is now handled at the requested type, which benefits speculative decoding setups. Fixes for Qwen 3.5 include corrected expert quantization handling and gathering of packed gate_up weights in a single launch, reducing GPU kernel overhead.
Server-side improvements include fixing a race condition on the scheduler's loaded map and hardening flaky updater/transfer unit tests. On the client side, an agent permission skill loading fix and a new command for setting agent system prompts via the TUI were added. The release also introduces Laguna MLX support for Apple Silicon users, ensuring loaded models stay memory resident. Assets for multiple platforms are available. This pre-release is intended for testing; users are encouraged to report issues.
- lm_head now quantized to 8-bit in requested families for better memory usage
- Qwen 3.5 expert quantization and gate_up packing fixes improve performance
- Laguna MLX support added; draft model output head quantization at requested type
Why It Matters
Better quantization control and hardware support make local LLMs more efficient for developers and power users.