Developer Tools

llama.cpp v10051 adds ARM SME v1 vs SME2 distinction for correct kernel dispatch

New release fixes incorrect use of SME2 instructions on SMEv1-only ARM hardware.

Deep Dive

The latest release of llama.cpp (b10051) addresses a critical issue in ARM Scalable Matrix Extension (SME) kernel dispatch. Prior to this fix, the codebase treated all SME support as a single capability, but the dispatched kernels relied on SME2-specific instructions. This caused hardware without SME2 (i.e., SMEv1-only processors) to fail or produce incorrect results, as the kernel assumed instructions unavailable on such chips. The bug was particularly impactful as ARM's SME architecture is becoming common in AI accelerators and mobile processors.

This release introduces both compile-time flags and runtime CPU feature detection to distinguish between SME(v1) and SME2. Kernels are now properly selected based on the actual hardware capabilities. The change, contributed under the KleidiAI integration, ensures that llama.cpp runs correctly across the entire ARM ecosystem. For developers deploying local LLMs on ARM devices—from Raspberry Pis to high-end mobile chips—this update guarantees stability and performance without manual workarounds. The patch is a model of how to handle evolving ISA extensions in cross-platform AI software.

Key Points
  • Version b10051 distinguishes ARM SME(v1) vs SME2 in kernel dispatch to prevent incorrect instruction use.
  • Previously, all SME kernels used SME2 instructions, crashing or miscomputing on SMEv1-only hardware.
  • Build-time and runtime detection now selects the correct kernel based on actual CPU feature support.

Why It Matters

Ensures correct ARM AI inference on mixed SME hardware, improving compatibility for edge devices.

📬 Get the top 10 AI stories daily