llama.cpp b9985 adds fused top-k MoE with SYCL GPU acceleration
New release optimizes mixture-of-experts models on Intel GPUs via SYCL
Deep Dive
ggml-org/llama.cpp released a new version (tag b9985, July 13) adding fused top-k mixture-of-experts (MoE) with SYCL support, co-authored with Anthropic's Claude Fable 5. The release includes builds for macOS, Linux, Windows, Android, and more platforms.
Key Points
- New fused top-k MoE kernel reduces memory transfers and speeds up sparse expert models
- SYCL backend enables GPU acceleration on Intel GPUs (integrated and discrete) and other cross-platform hardware
- Co-authored by Anthropic's Claude Fable 5, showcasing AI-assisted development in open source
Why It Matters
Enables faster MoE inference on Intel GPUs, broadening hardware options for local LLM deployment and reducing dependency on NVIDIA CUDA.