llama.cpp b10098 optimizes Hexagon activation ops for faster on-device AI

The latest release of llama.cpp (b10098) focuses on optimizing Hexagon DSP performance for running large language models locally on Qualcomm-powered devices. Key updates include a revamped activation ops system that now supports non-contiguous source tensors and strided DMA transfers, significantly reducing memory bottlenecks during inference. The release also introduces an all-in-one geglu microkernel designed for Hexagon, which consolidates multiple operations into a single, efficient kernel to minimize overhead and latency. These improvements are built on top of per-thread GLU functions and a local `htp_vtcm_layout` computation for better scratchpad management.

This release underscores llama.cpp’s commitment to edge AI performance. By tailoring kernel code to Hexagon’s vector architecture, the project achieves lower latency and higher throughput for transformer-based models on mobile and embedded devices. The b10098 update ships with pre-built binaries for all major platforms, including macOS (Apple Silicon and Intel), Linux (x64, ARM64, s390x), Windows (CPU, CUDA, Vulkan, OpenCL), Android, and iOS. For developers and researchers deploying LLMs on Qualcomm hardware, this release represents a meaningful step toward practical on-device inference without compromising speed or accuracy.