llama.cpp v9605 adds CUDA concat for scalar types
New release enables efficient tensor concatenation for scalar types on CUDA.
The open-source llama.cpp project, led by ggml-org, has released version b9605, introducing support for concatenating scalar tensors on the CUDA backend. This fixes a limitation where scalar types (e.g., tensors of shape [1,1]) could not be efficiently concatenated, impacting certain model operations and layer combinations. The update also resolves a CI issue with Metal builds on macOS/iOS, ensuring smoother cross-platform compatibility.
The release is distributed as pre-built binaries for Windows (CPU, ARM64, CUDA 12/13, Vulkan), Linux (x64, ARM64, s390x), macOS (Apple Silicon & Intel), and Android (ARM64). For developers running AI models locally, this fix improves flexibility when working with models that require scalar tensor joins, such as in custom attention mechanisms or quantized layer operations. The update is part of llama.cpp's ongoing optimization for running large language models on consumer hardware.
- New support for concatenating scalar tensors on the CUDA backend
- Includes a fix for Metal CI on macOS/iOS builds
- Available across 11+ platform variants including Windows ARM64 and Android
Why It Matters
Enables more efficient tensor operations for local AI models, improving on-device inference flexibility.