llama.cpp v10178 adds trace logging for slot similarity checking
New release helps debug prompt caching with detailed slot similarity metrics.
llama.cpp, the popular C/C++ implementation of LLM inference, released version b10178 with a focused improvement for its server component. The update adds trace-level logging for slot similarity checking during prompt cache slot selection. This means developers can now see exactly why certain cache slots are skipped and how similarity scores are calculated, aiding in debugging and performance tuning.
The release includes builds for macOS (Apple Silicon, Intel, iOS), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA, Vulkan, OpenVINO, SYCL, HIP), and Android. The trace logging feature, assisted by llama.cpp's own Qwen3.6-27B model, provides visibility into the caching logic that optimizes memory and latency when serving multiple concurrent requests. For teams using llama.cpp in production, this small but impactful update simplifies optimization of prompt caching strategies.
- Trace logging added for slot similarity checking in server-context.cpp
- Includes skip reasons and similarity calculation details for prompt cache
- Assisted by llama.cpp's Qwen3.6-27B model during development
Why It Matters
Better visibility into prompt caching behavior means faster tuning and lower latency for llama.cpp server deployments.