llama.cpp b10159 adds FWHT kernel for faster Metal performance

The latest release of llama.cpp, version b10159, introduces a significant performance optimization for Apple users: a Fast Walsh-Hadamard Transform (FWHT) kernel for the Metal backend. This kernel, contributed by YiChen Lv and merged by Georgi Gerganov, accelerates the matrix multiplications that underpin transformer models. By leveraging the FWHT algorithm, the Metal backend can achieve higher throughput on Apple Silicon (M-series) and Intel Macs, especially for large batch sizes or attention-heavy layers. The release also includes precompiled binaries for macOS (ARM64 and x64), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android (ARM64), and Windows (CPU, CUDA, Vulkan, OpenVINO, etc.), making it easy for developers to deploy locally.

This update underscores llama.cpp's ongoing commitment to edge AI performance. The FWHT kernel is particularly beneficial for running large language models like Llama 3, Mistral, or Mixtral on consumer hardware without a GPU server. Users can expect smoother real-time interactions and lower latency for chat interfaces or local assistants. While the change is incremental, it reflects a broader trend: specialized kernels for diverse backends (Metal, Vulkan, CUDA) are unlocking near-native speeds for LLMs on personal devices. For developers and enthusiasts who rely on llama.cpp for privacy-preserving AI, this release is a welcome step toward bridging the gap between local and cloud performance.