b8278

The open-source llama.cpp project, maintained by ggml-org, has released a critical patch in version b8278. This update addresses a regression in the quantization logic that was introduced in pull request #19770. The bug was in the `quantize_state_impl` function, where the `n_attention_wv` counter was being incorrectly incremented and used within the same loop. This faulty logic affected the `llama_tensor_get_type_impl` function's decision-making for the `use_more_bits` parameter, potentially leading to the wrong quantization bits being assigned to specific tensors during the model conversion process.

The significance of this fix lies in its subtlety. The bug's author noted they "never observed a difference" in their own tests; it was only discovered after contributor @bartowski identified the issue. This highlights how quantization errors can be silent, corrupting model outputs without causing immediate crashes or obvious failures. The patch corrects the counter initialization, ensuring the tensor type is fixed before being evaluated for bit allocation. The release includes pre-built binaries for a wide range of platforms, from macOS Apple Silicon and Windows CUDA to Linux ROCm and openEuler, making this stability fix universally available for developers running quantized Llama models locally.