Developer Tools

llama.cpp v9850 adds Qwen3Next support with attention normalization fix

New llama.cpp release brings Qwen3 compatibility and performance fixes for local AI.

Deep Dive

The ggml-org/llama.cpp project has released version b9850, a significant update that brings enhanced support for the Qwen3 model family. This release registers the t_layer_inp tensor specifically for Qwen3Next models, fixes input assignment in the layer processing loop, resolves DFLASH issues for qwen-coder-next, and adds a new tensor for attention normalization in the Qwen3 architecture. These changes ensure compatibility with the latest Qwen3 variants while fixing critical bugs in the inference pipeline.

The release comes with pre-built binaries across a wide range of platforms: macOS (Apple Silicon, Intel, iOS), Linux (x64, arm64, s390x with Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and Android (arm64 CPU and OpenCL Adreno). This broad support allows developers and enthusiasts to run Qwen3 models locally on virtually any modern device. The update continues llama.cpp's mission to bring state-of-the-art LLM inference to consumer hardware without cloud dependencies.

Key Points
  • Registers t_layer_inp tensor for Qwen3Next models to ensure correct layer processing
  • Fixes DFLASH for qwen-coder-next and adds attention normalization tensor for Qwen3
  • Pre-built binaries available for 20+ platforms including macOS, Linux, Windows, and Android

Why It Matters

Enables local deployment of latest Qwen3 models with improved stability and performance.

📬 Get the top 10 AI stories daily