llama.cpp b9661 adds Vulkan col2im_1d operation for neural networks
New release improves GPU acceleration with col2im_1d for convolution backpropagation on Vulkan.
The latest llama.cpp release (b9661) introduces support for the col2im_1d operation on the Vulkan backend, a key building block for convolutional neural network (CNN) workloads. The col2im (column-to-image) operation is essential for algorithms like im2col, commonly used in convolution forward and backward passes. By adding the GGML_OP_COL2IM_1D op directly in Vulkan, llama.cpp can now execute these operations entirely on GPU without CPU fallback, improving performance for models that incorporate convolution layers (e.g., vision transformers, multi-modal LLMs). The implementation uses a bounded gather loop instead of a full-K scan with modulo, optimizing memory access patterns for Vulkan compute shaders.
This release also expands platform support significantly, providing precompiled binaries for macOS (Apple Silicon arm64 and Intel x64, including KleidiAI-optimized arm64), Linux (x64, arm64, s390x with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (x64 CPU/arm64 CPU, CUDA 12/13, Vulkan, SYCL, HIP), and Android (arm64 CPU). The UI was also updated. As llama.cpp remains the go-to solution for running large language models locally on consumer hardware, this low-level optimization helps democratize access to advanced AI by enabling faster inference and training on Vulkan-compatible GPUs across operating systems.
- llama.cpp version b9661 adds the col2im_1d operation (GGML_OP_COL2IM_1D) to the Vulkan backend, improving GPU acceleration for convolution layers.
- The release was led by GitHub contributors jeffbolznv and 0cc4m, using a bounded gather loop for efficient GPU memory access.
- Prebuilt binaries are available for macOS, Linux, Windows, Android, and multiple GPU backends (Vulkan, CUDA 12/13, ROCm, SYCL, OpenVINO, HIP).
Why It Matters
Enables faster local inference for multi-modal and vision LLMs on Vulkan GPUs across platforms.