llama.cpp b9558 speeds up Vulkan GPU inference with vec4 loads

The llama.cpp project, a C++ implementation of LLaMA model inference, just shipped b9558 with a targeted Vulkan backend optimization. The key change in pull request #23991 modifies how matrix multiplication handles the B matrix in mul_mat_id operations. Instead of loading B elements one-by-one, the new code uses the cm2 decode_vector instruction to perform vec4 (four-element) loads. Additionally, the internal block size BK is increased to 64 when this feature is enabled. The developers note that neither change alone is consistently faster, but together they provide a ‘nice speedup’.

This update is part of an ongoing effort to improve GPU utilization for running large language models locally. Vulkan support has been a major focus for llama.cpp, as it allows cross-platform acceleration on AMD, NVIDIA, and Intel GPUs without vendor-specific CUDA or ROCm dependencies. The b9558 release includes pre-built binaries for macOS (Apple Silicon and Intel), Linux (x64/arm64 with Vulkan, ROCm, OpenVINO, SYCL), Windows (x64/arm64 with CUDA, Vulkan, HIP), Android (arm64 CPU), and iOS. The commit was signed and verified, ensuring code integrity.