llama.cpp b9891 adds Metal col2im_1d op for Apple Silicon speed boost
New Metal kernel speeds up 1D column-to-image operations on Mac, iOS devices.
The latest release of llama.cpp, version b9891, introduces a critical performance optimization for Apple hardware users. The standout feature is the addition of a col2im_1d operation for the Metal backend, supporting f32, f16, and bf16 data types. This kernel mirrors the design of existing CPU and CUDA implementations: each output element reads its ceil(K/s0) source columns using an F32 accumulator, with linear indexing and a single write—no atomics required. The implementation uses one thread per output element and 256 threads per threadgroup, which aligns with Metal's GPU architecture on Apple Silicon.
Beyond the new kernel, the release includes a support operation predicate for GGML_OP_COL2IM_1D that now checks dst contiguity and type matching, ensuring the Metal backend behaves consistently with CPU, CUDA, and Vulkan backends. This change prevents silent errors when the destination tensor is non-contiguous or mismatched in data type. The update is part of ongoing work to bring feature parity and performance improvements across all llama.cpp backends, with a focus on optimizing local LLM inference on macOS, iOS, and other Metal-compatible devices.
- New col2im_1d Metal kernel supports f32, f16, and bf16 data types on Apple Silicon
- Kernel uses linear indexing with one thread per output element and 256 threads per threadgroup
- Added contiguity and type-matching checks for Metal's col2im_1d operation to align with CPU/CUDA/Vulkan backends
Why It Matters
Speeds up local LLM inference on Apple Silicon by optimizing a key neural network operation with Metal.