llama.cpp b10034 fixes MoE model corruption on Adreno A7x GPUs
Mixture-of-Experts models on Adreno A7x now generate correct outputs
ggml-org's llama.cpp release b10034 addresses a long-standing bug affecting mixture-of-experts (MoE) models on Adreno A7x GPUs. The issue stemmed from certain OpenCL compilers miscompiling the repack kernels, which corrupted model weights and produced garbage output during inference. This fix explicitly excludes the problematic kernels from MoE weights repack on Adreno A7x and unknown Adreno devices, restoring correct behavior. The release also includes updates for ARM64, x64, and various GPU backends (CUDA, Vulkan, ROCm, OpenVINO, SYCL, HIP), with builds for Windows, Linux, macOS, iOS, and Android.
For developers and users running large language models locally using llama.cpp, this fix is critical for accurate MoE inference on mobile and embedded devices with Adreno A7x GPUs. The bug was particularly insidious because it silently produced plausible-sounding nonsense. b10034 ensures that models like Mixtral or Qwen-MoE generate correct outputs on these popular mobile GPUs. The release also enables KleidiAI on macOS Apple Silicon and provides separate CUDA builds for CUDA 12.4 and 13.3, expanding compatibility.
- Release b10034 fixes MoE kernel miscompilation on Adreno A7x devices, which corrupted weights and caused garbage output
- Fix excludes the repack kernels from MoE weights repack on A7x and unknown Adreno devices
- Release includes builds for multiple platforms: Windows (CPU, CUDA, Vulkan, HIP), Linux, macOS, Android, iOS
Why It Matters
Ensures correct inference of mixture-of-experts LLMs on Adreno A7x mobile GPUs, critical for local AI.