b8949
New release fixes RPC cache bug on Windows and expands build support.
The llama.cpp project, a popular open-source library for running large language models locally, released version b8949. This update addresses a critical bug where the RPC server cache was not functioning correctly in Windows environments. The fix ensures reliable caching, which is essential for performance when running models across multiple machines. Additionally, the release improves directory creation and log cache file naming, removing conditional compilation for GGML_LOG_INFO to simplify the codebase.
Version b8949 also expands build support to a wide range of platforms, including macOS (Apple Silicon, Intel, iOS XCFramework), Linux (x64, arm64, s390x, with support for Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android (arm64), Windows (x64, arm64, with CUDA 12/13, Vulkan, SYCL, HIP), and openEuler (x86 and aarch64 with ACL Graph). This broad compatibility allows developers to deploy LLM inference across diverse hardware, from local desktops to cloud servers.
- Fixes RPC server cache bug in Windows environments for reliable multi-machine inference.
- Adds directory creation and log cache file naming improvements, removing conditional compilation.
- Expands build support to macOS, Linux, Android, Windows, and openEuler with various GPU backends.
Why It Matters
Ensures stable local LLM deployment across platforms, critical for developers running models offline.