llama.cpp's b10505 adds dedup cache for leaner server memory

The llama.cpp project, maintained by ggml-org, rolled out release b10505 on August 20th, headlined by a new server option called dedup-cache-models. This feature, tracked in PR #27346, gives developers a preset to deduplicate cache models inside the llama.cpp server. While detailed documentation is still sparse, the intent is clear: eliminate redundant cache entries that can balloon RAM usage when loading or switching between multiple models. For teams running llama.cpp as a lightweight inference backend — often on edge devices or multi-tenant GPU boxes — this can translate into meaningful memory savings and potentially higher throughput.

The release is especially notable because llama.cpp remains the de facto standard for local LLM inference, with 125,000 stars and 21,900 forks. The b10505 release package includes prebuilt binaries across a wide range of hardware targets: macOS (Apple Silicon and Intel, with optional KleidiAI), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL, and even s390x), Windows (CUDA 12 and 13, Vulkan, OpenCL Adreno, OpenVINO, SYCL), plus Android arm64 and iOS XCFramework builds. OpenEuler variants also appear for Ascend NPU users. This breadth of platform support reinforces llama.cpp's role as the go-to engine for deploying LLMs on almost any hardware. For developers already using the server API, upgrading to b10505 and enabling the dedup-cache-models preset is a low-risk way to test potential memory improvements in their own inference pipelines.