Llama.cpp b9768 adds local inference for IBM's Granite Speech Plus model

The llama.cpp project has released version b9768, a significant update that adds first-class support for IBM's Granite Speech Plus model. This open-source C++ library, already a favorite for running large language models locally, now extends its capabilities to multi-modal speech AI. The release includes conversion scripts (feat: Add conversion support for Granite Speech Plus) and architectural improvements such as multi-layer concatenation for audio feature extraction, enabling the model to handle complex speech tasks. Notably, the release is signed and verified, with contributions from IBM engineer Gabe Goodhart and community collaborators.

The build matrix is impressively broad: macOS users get native Apple Silicon (arm64) binaries, including a KleidiAI-optimized variant, plus Intel x64 and iOS XCFramework support. Linux users can run on CPU (x64, arm64, s390x) or accelerate with Vulkan, ROCm 7.2, OpenVINO, or Intel SYCL (FP32/FP16). Windows builds cover CPU (x64, arm64), CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP for AMD GPUs, and even OpenCL Adreno for Qualcomm mobile GPUs. Android arm64 is also supported. This means developers can deploy Granite Speech Plus on everything from desktop workstations to mobile devices, all without any cloud round-trip.