llama.cpp b9624 speeds up UI loading with gzip compression
Lighter UI assets and broader GPU support make local LLM inference more efficient...
The open-source llama.cpp project, known for running LLaMA-family models locally, released version b9624 with significant UI and platform enhancements. The headline feature is build-time gzip compression for UI assets, which reduces the size of shipped files and thus decreases loading times for the bundled web interface. A related fix addresses the nocache UI compression issue, ensuring assets are properly compressed when caching is disabled. Additionally, the update preserves original file names and paths, simplifying debugging and asset management.
Hardware support expands considerably in this release. New prebuilt binaries target CUDA 12.4 and 13.3 with bundled DLLs, as well as ROCm 7.2, OpenVINO, SYCL FP16 and FP32, and Vulkan for multiple architectures. macOS gets KleidiAI support for Apple Silicon (enabled separately), while Linux adds s390x builds. iOS and Android arm64 are also included. These additions let developers and enthusiasts run llama.cpp on a wider variety of CPUs and GPUs without compiling from source. The release reinforces llama.cpp's position as the most versatile tool for local AI inference.
- UI assets now use build-time gzip compression, reducing load times by up to 50% for the web interface
- New prebuilt binaries for CUDA 12.4/13.3, ROCm 7.2, OpenVINO, SYCL FP16/FP32, and s390x Linux
- KleidiAI support added for Apple Silicon (disabled by default) to leverage hardware acceleration
Why It Matters
Local AI inference gets faster and more accessible across GPU vendors, lowering the barrier for self-hosted LLMs.