llama.cpp release b9937 fixes CUDA snake fusion and adds contiguity guard
New release aligns CUDA matcher with CPU/Metal, preventing incorrect F16/BF16 reads.
The llama.cpp project released b9937 with several critical fixes for the CUDA backend's snake fusion implementation. The matcher now aligns with CPU and Metal backends by requiring that the input type matches the expected F32 format for the `a` and `inv_b` buffers. Previously, F16/BF16 chains would never fuse and silently fell back to a slower naive path, while a hypothetical all-F16 chain could read F16 bits as float, producing incorrect results.
Additionally, the new release adds a contiguity guard to the CUDA snake fusion matcher – mirroring guards already present in the CPU, Vulkan, and Metal backends. The kernel reads linearly from `x[idx]`, `a[c]`, and `inv_b[c]`, so a non-contiguous view passing the matcher would silently read wrong data. The release also includes build artifacts for multiple platforms: macOS (Apple Silicon and Intel), Linux (x64, arm64, s390x), Windows (x64, arm64), Android, iOS, and various GPU backends like CUDA, Vulkan, ROCm, OpenVINO, SYCL, and OpenCL.
- CUDA snake fusion matcher now requires matching types between input and float buffers, fixing fallback to naive path for F16/BF16
- Added contiguity guard for CUDA snake fusion kernel, preventing silent data corruption from non-contiguous tensors
- Release includes prebuilt binaries for macOS, Linux, Windows, Android, iOS, CUDA 12/13, Vulkan, ROCm 7.2, OpenVINO, SYCL, and more
Why It Matters
Ensures correct and efficient LLM inference on NVIDIA GPUs, closing a performance and correctness gap in llama.cpp.