TurboFieldfare runs Gemma 4 26B on 2 GB RAM on any M-series Mac

TurboFieldfare is a model-specific open-source engine built entirely in Swift and Metal, designed to run Google's new Gemma 4 26B-A4B instruction-tuned model on any Apple Silicon Mac, even those with only 8 GB of RAM. Instead of loading the full 14.3 GB model into memory, it intelligently keeps a shared 1.35 GB core and FP16 KV cache resident, then streams only the MoE experts needed for each token directly from the SSD. This novel approach brings a 26B-parameter model (3.88B active per token) to memory-constrained devices. The engine uses 4-bit affine quantization (group size 64) for weights, with an 8-bit router. Total memory footprint is about 2 GB for weights plus the KV cache (default 4K context).

On an 8 GB M2 MacBook Air, TurboFieldfare delivers 5.1-6.3 tokens per second for decode; on a 24 GB M5 Pro, it reaches 31-35 tok/s. The package provides a native macOS app, a command-line interface, and an experimental loopback OpenAI-compatible server that supports function-tool declarations. Installation is via git clone and Swift build; the app handles downloading and repacking the model (~15 GB) from Hugging Face. The runtime requires macOS 26 with Metal 4 and Swift 6.2. This project opens up local, private inference of a capable open-weight model on everyday Apple hardware without cloud costs.