llama.cpp v9568 adds Gemma-4 E2B/E4B assistant model support
Google's Gemma-4 now runs locally with multi‑turn assistants in llama.cpp.
Deep Dive
The latest llama.cpp release (b9568) adds support for Gemma‑4 E2B and E4B assistants, updates the model converter to handle smaller assistants, and adds `masked_embd` tensors to the `gemma4-assist` architecture. Builds are available for macOS (Apple Silicon, Intel), Linux (x64, arm64, s390x with Vulkan/ROCm/OpenVINO), Windows (x64/arm64 with CUDA/Vulkan/HIP), and Android.
Key Points
- Adds experimental support for Gemma‑4 E2B and E4B assistant models (PR #24282).
- Updates model converter to handle smaller assistant models and filters out masked_embd tensors during conversion.
- Builds available for macOS, Linux (x64/arm64/s390x with Vulkan/ROCm/OpenVINO), Windows (x64/arm64 with CUDA/Vulkan/HIP), and Android arm64.
Why It Matters
Enables local, offline inference of Google’s latest Gemma‑4 assistants, giving developers more control and privacy.