b8998
New release enables unary ops on Hexagon across 20+ platforms.
The llama.cpp project has released version b8998, a significant update that enables non-contiguous row tensor support for unary operations on Hexagon processors. This improvement allows the popular open-source C/C++ LLM inference engine to leverage Qualcomm's Hexagon DSP more effectively, expanding the range of operations that can be offloaded to this low-power accelerator. The change is part of ongoing efforts to optimize local AI inference on edge and mobile hardware.
This release is accompanied by an extensive set of pre-built binaries covering macOS (Apple Silicon and Intel), Linux (x64, arm64, s390x, plus GPU backends like Vulkan, ROCm 7.2, OpenVINO, and SYCL), Windows (x64, arm64, CUDA 12/13, Vulkan, SYCL, and HIP), Android (arm64), and openEuler. With over 108,000 stars and 17,700 forks, llama.cpp continues to be a cornerstone of local AI deployment, and b8998 broadens its reach into more embedded and mobile use cases.
- Adds non-contiguous row tensor support for unary ops on Hexagon processors (Commit #22574).
- Pre-built binaries cover macOS, Linux, Windows, Android, and openEuler with multiple GPU backends.
- Project has 108k stars and 17.7k forks, reflecting widespread adoption for local LLM inference.
Why It Matters
Expands local LLM inference to Qualcomm Hexagon hardware, enabling efficient AI on edge devices.