b8144

The ggml-org team behind the massively popular llama.cpp project has released a significant update with commit b8144. This commit, part of the ongoing development of the framework that enables efficient local execution of models like Llama 3 and others, introduces a crucial API refinement for developers. The core change is the support for a new `max_completion_tokens` request property in the server component, formally deprecating the previous `max_tokens` parameter. This technical shift provides more precise control over generation limits by explicitly defining the maximum number of tokens for the entire completion process, which includes any internal reasoning steps plus the final text output.

The update, which closes long-standing GitHub issue #13700, addresses a common pain point for developers building applications on top of llama.cpp's server. The old `max_tokens` parameter could lead to ambiguity, especially with models that utilize chain-of-thought or internal reasoning. The new property establishes a clear, hard upper bound for the total token count of a response, improving predictability and resource management for production deployments. This is a backend change for the `/completion` endpoint, meaning existing client code using `max_tokens` will continue to work but developers are encouraged to migrate.

The release is packaged and available across llama.cpp's extensive multi-platform support matrix. Pre-built binaries are offered for macOS (both Apple Silicon and Intel), various Linux distributions (including CPU, Vulkan, and ROCm backends), Windows (with support for CPU, CUDA 12/13, Vulkan, SYCL, and HIP), and iOS. This ensures the update is immediately accessible to the project's vast user base, from researchers on Linux clusters to consumers running models on personal Apple devices. The change underscores the project's maturation from a research tool into a stable platform for deploying local AI applications, where predictable resource usage is paramount.