Developer Tools

Llama.cpp b9631 fixes token preservation, expands platform builds

New release patches CLI token copying bug across all major platforms.

Deep Dive

The llama.cpp project, a widely-used open-source C/C++ implementation for running LLaMA-family large language models locally, has shipped version b9631. The headline fix addresses a bug in the command-line interface where tokens marked as 'preserved' were not properly copied, as reported in issue #24258. This could lead to unexpected behavior when applying token-level controls (e.g., repetition penalties or special token handling). The patch ensures that preserved tokens are faithfully retained during model inference, improving output accuracy for advanced users.

For deployment flexibility, b9631 provides over a dozen precompiled binaries. Mac users gain builds for Apple Silicon (both standard and KleidiAI-enabled), Intel, and an iOS XCFramework. Linux builds include CPU-only for x64/arm64/s390x, Vulkan, ROCm 7.2, OpenVINO, and SYCL (FP32/FP16). Windows users get CPU, ARM64, CUDA 12/13, Vulkan, SYCL, and HIP variants. The release also includes an Android arm64 binary and UI assets. This broad platform support makes the fix instantly available to developers and hobbyists running LLMs on diverse hardware, from gaming PCs to edge devices.

Key Points
  • Fixes CLI bug where preserved tokens weren't copied (issue #24258), improving token fidelity
  • Prebuilt binaries for macOS, Linux, Windows, Android, and iOS, covering CPU, CUDA, Vulkan, ROCm, OpenVINO, and SYCL
  • Includes community-maintained UI assets alongside core inference library updates

Why It Matters

A stable token pipeline is essential for reliable LLM output in local AI applications and development workflows.

📬 Get the top 10 AI stories daily