llama.cpp b9689 boosts Apple Silicon with f16/bf16 concat support
Metal backend now handles half-precision tensors for concat, speeding local LLMs on Macs.
Llama.cpp, the popular open-source C/C++ library for running large language models locally, has released version b9689 with a significant update to its Metal backend. This release extends the concat operator to support f16 (half-precision) and bf16 (bfloat16) tensor types, in addition to the existing f32 and i32 support. The concat operation is a core building block in transformer architectures, used in attention mechanisms and layer merging, so optimizing it for lower precision directly accelerates inference on Apple's GPU.
The implementation uses a template kernel `kernel_concat` specialized for `float`, `half`, `bfloat`, and `int` types. A new type-specific pipeline getter `ggml_metal_library_get_pipeline_concat()` was added, and device support checks now allow f16 unconditionally and bf16 where the Metal device supports bfloat16. This work was partially assisted by llama.cpp's own Qwen3.6-27B AI agent. The update improves performance across all Apple Silicon targets (arm64 macOS, iOS, XCFramework) and is compatible with existing CPU, Vulkan, and CUDA builds on other platforms.
- Metal backend concat operator now supports f16 and bf16 tensor types for faster local LLM inference.
- Release b9689 uses template kernel specializations and device-specific pipeline getters.
- Partially assisted by llama.cpp's Qwen3.6-27B AI agent for development.
Why It Matters
Half-precision Metal concat means faster, more memory-efficient LLMs on Macs and iOS devices.