llama.cpp b10189 simplifies Apple M3 support with stock ops
New llama.cpp release strips custom M3 CPU op for cleaner, faster code
llama.cpp, the popular open-source project for running large language models locally, has released version b10189, released on July 30. The headline change is the removal of a custom CPU op from the Apple M3 graph, replacing it with standard stock operations (PR #26297). This architectural cleanup reduces code complexity and aligns M3 support with the project's core codebase, potentially improving performance and maintainability for Apple Silicon users running models on-device.
The release also comes with an expanded set of prebuilt binaries and platform support. Builds are available for macOS (incl. KleidiAI-enabled arm64), iOS, Linux (x64, arm64, s390x, Vulkan, ROCm 7.2, OpenVINO, SYCL FP16/FP32), Windows (x64/arm64 CPU, CUDA 12/13, Vulkan, OpenCL Adreno, SYCL, HIP), and openEuler. This ensures developers and end-users can easily update their local LLM setup across a wide range of hardware.
- PR #26297 removes custom CPU op from Apple M3 graph, using stock ops for simplification
- Supports 30+ platform/backend combinations including CUDA 12/13, Vulkan, ROCm, and OpenVINO
- Includes KleidiAI-enabled macOS arm64 builds for accelerated inference
Why It Matters
Streamlines open-source LLM deployment on M3 Macs, making local inference more reliable across platforms.