llama.cpp b9127 adds Adreno GPU optimization for faster prefill

llama.cpp, the popular open-source C++ implementation of LLaMA, released version b9127 with a significant performance optimization for Qualcomm Adreno GPUs. The key addition is an opt-in Adreno xmem F16xF32 GEMM (general matrix multiply) kernel designed specifically for the prefill phase of LLM inference. Prefill is the initial processing of input tokens, which can be a bottleneck on mobile and edge devices. By leveraging Adreno's xmem capabilities, this kernel allows mixed-precision (F16 input, F32 accumulation) computation, potentially leading to faster prompt processing on a wide range of Qualcomm-powered devices.

The release includes comprehensive platform support: macOS (Apple Silicon, Intel, iOS), Linux (x64/arm64 with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Android (arm64 CPU), and Windows (x64/arm64 with CPU, CUDA 12/13, Vulkan, SYCL, HIP). This broad compatibility ensures developers can deploy the optimization across diverse environments. As an opt-in feature, it requires explicit enabling, likely through compile-time flags or runtime settings. The update aligns with the growing trend of running LLMs locally on consumer hardware, particularly on mobile devices where power-efficiency and latency are critical.