llama.cpp b10273 fixes sampler context: -1 now equals 1024 tokens
New llama.cpp release changes history-based sampler defaults to 64 tokens, improving stability.
llama.cpp, the popular C/C++ inference engine for LLaMA models, dropped release b10273. The update addresses a tricky interaction between backend sampling and context length. Previously, samplers could use 'full-context windows' by inferring the context size from the llama_context object. But with backend-sampling, samplers are now constructed before the complete llama_context is available, making it impossible to know the eventual context length. To fix this, the project removed full-context windows from history-based samplers and introduced shared defaults: 64 tokens for history-based samplers, and -1 now resolves to 1024 tokens instead of the context length.
This change affects how samplers like repetition penalty, top-k, and others use historical tokens. Developers no longer need to rely on context size when configuring sampling parameters, which simplifies setup and ensures more consistent behavior across different models and backends. The release also includes builds for a wide range of platforms: macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenCL Adreno), Android, and openEuler with various accelerators. For a project with 123k stars and 21.3k forks, this minor version tweak brings meaningful reliability to local LLM generation.
- Removes 'full-context windows' from history-based samplers due to backend-sampling initialization timing.
- -1 now resolves to 1024 tokens for samplers, replacing the previous ctx-len behavior.
- Shared default of 64 tokens for history-based samplers, with builds for macOS, Linux, Windows, Android, and more.
Why It Matters
This fix ensures predictable token sampling across backends, making local LLM output more stable for developers.