Ollama v0.31.2 enables flash attention on older NVIDIA GPUs
Ollama's latest update supercharges older NVIDIA cards with flash attention and fixes thinking models.
Ollama v0.31.2, released on July 6, 2025, delivers several performance and compatibility upgrades. The headline feature is flash attention support for older NVIDIA GPUs with compute capability 6.x (e.g., Maxwell and Pascal architectures). This optimization reduces memory bandwidth usage and speeds up attention computation, making it practical to run larger models on older hardware. Additionally, integrated GPUs can now offload vision models by dynamically padding memory, enabling users with limited VRAM to run multimodal models like LLaVA locally.
Other fixes include a correction for structured output generation in thinking models when the thinking feature is disabled — a boon for developers relying on JSON mode. GGUF model creation is now hardened against corruption, and the `ollama launch` command for Claude Code disables telemetry by default for privacy. The update also resolves a bug affecting model loading from paths with non-UTF-8 characters and bumps the underlying MLX and llama.cpp engines for better performance and stability.
- Flash attention now supported on older NVIDIA GPUs (compute capability 6.x) for faster inference.
- Integrated GPUs can offload vision models by padding memory to fit available VRAM.
- Fixed structured output for thinking models when thinking is disabled, and hardened GGUF creation.
Why It Matters
Extends Ollama's local AI capabilities to older hardware and fixes critical output bugs for developers.