Llama.cpp b9693 brings BF16 Metal kernel for Apple Silicon
New release speeds up LLM inference on Macs with BF16 concat support.
Deep Dive
llama.cpp's latest release, b9693, includes a change to check for BF16 support in the Metal concat kernel. Builds are available for macOS, iOS, Linux, Windows, Android, and backends like CUDA and Vulkan.
Key Points
- New version b9693 adds BF16 support in the Metal concat kernel for Apple Silicon GPUs.
- Supports macOS, iOS, Linux (CPU/Vulkan/ROCm/OpenVINO/SYCL), Windows (CPU/CUDA/Vulkan/HIP), and Android arm64.
- llama.cpp is a top open-source LLM runner with 117k GitHub stars and 19.7k forks.
Why It Matters
Enables faster, more memory-efficient local LLM inference on millions of Apple devices.