llama.cpp v0.0.0-b10084 brings Hexagon DSP support for Qualcomm devices

llama.cpp, the leading open-source library for running LLaMA-family large language models locally on consumer hardware, has released version b10084. The most notable change is a commit titled “hexagon: check tensor type when reusing descriptors (#25968)”. This fix optimizes memory management and tensor operation scheduling on Qualcomm’s Hexagon DSP, a specialized processor found in many Snapdragon chips used in smartphones and IoT devices. By ensuring tensor descriptors are correctly reused, the update reduces overhead and improves inference speed for local AI workloads on these mobile processors.

The release also bundles precompiled binaries across an extensive range of platforms: macOS (Apple Silicon and Intel), Linux (x64, arm64, s390x, plus build options for Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Windows (x64, arm64, with CUDA 12/13, Vulkan, OpenVINO, SYCL, and HIP for AMD GPUs), Android arm64, and openEuler (x86 and aarch64 with ACL). This comprehensive support list makes it easy for developers to integrate local inference into applications on almost any modern device. The fix specifically benefits mobile developers building AI chatbots or document assistants that run entirely on-device using Qualcomm’s AI accelerators.