llama.cpp b9581 cuts Vulkan memory use for faster inference

The latest release of llama.cpp (b9581), a popular open-source framework for running large language models locally, introduces a targeted Vulkan optimization: reduced shared memory usage for iq1 quantized matrix multiplication (mul_mm). IQ1 is an aggressive 1-bit quantization scheme that allows running large models on limited hardware, but it often runs into shared memory bottlenecks on GPUs. By trimming the shared memory footprint of the iq1 kernel, the update unlocks faster inference on GPUs with small shared memory banks, such as many mobile chipsets (Apple Silicon, ARM Mali) and entry-level discrete GPUs. The change is part of an ongoing effort to make local LLM deployment more practical on a wide range of devices.

Beyond the Vulkan tweak, the release demonstrates llama.cpp's cross-platform ambition. Build artifacts are provided for macOS (Apple Silicon with and without Kleidei AI, Intel x64), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, HIP), and Android (ARM64). Notably, CUDA 13 DLLs are included, catering to the latest NVIDIA drivers. The iOS XCFramework is also published. This breadth means developers and enthusiasts can drop in the update for their specific stack without recompiling. The release is signed with a verified GPG key, ensuring integrity.