update your llama.cpp - great tg speedup on Qwen3.5 / Qwen-Next

The open-source llama.cpp project, a popular C++ implementation for running large language models locally, has merged a critical optimization that dramatically speeds up inference for Alibaba's Qwen family of models. Contributor am17an's pull request (#19504) specifically targets the Qwen3.5 and upcoming Qwen-Next architectures, addressing inefficiencies in how the software processes these models' unique neural network layers. The fix, which primarily affects CUDA (NVIDIA GPU) and CPU computation backends, follows community testing that revealed suboptimal performance compared to other model families. This update represents a significant step in democratizing access to state-of-the-art Chinese-language AI, as Qwen models have gained recognition for their strong multilingual capabilities and coding proficiency.

The technical improvement, showcased in benchmark screenshots from Reddit user jacek2023, demonstrates a near doubling of token generation speed—from approximately 20 tokens per second to around 40 tokens per second on consumer hardware like the RTX 4090. This performance leap makes interactive use of the 72B parameter Qwen models far more practical for developers and researchers. The optimization likely involves better memory access patterns or kernel implementations for Qwen's specific architectural choices, such as its Qwen attention mechanism. For the open-source AI community, this update reduces the friction of experimenting with alternative model ecosystems beyond Meta's Llama and underscores the importance of collaborative optimization work. Users are advised to update their local llama.cpp installations to gain these speed benefits immediately.