Gemma 4 hits 255 tok/s in-browser via Fable 5's WebGPU kernels

Before its shutdown, AI optimization startup Fable 5 collaborated on fine-tuning WebGPU compute shaders for Google's Gemma 4 E2B model. The result: 255 tokens per second on an M4 Max Mac—near-native inference speed, entirely inside the browser. This performance was achieved through heavily optimized kernels that leverage WebGPU's low-level GPU access, bypassing typical WebGL or ONNX runtime bottlenecks. The model is Gemma-4-E2B-it-qat-mobile-transformers, a 4-billion-parameter variant designed for on-device inference with quantization-aware training (QAT). The release includes an interactive demo and the full kernel source code on Hugging Face Spaces.

With these kernels, developers can run a capable 4B-parameter LLM at 255 tok/s without any server-side compute—purely in the user's browser. This unlocks real-time, privacy-preserving applications like chat, code completion, and document analysis that never send data to the cloud. The speed rivals many local desktop apps, yet requires only a modern GPU with WebGPU support (e.g., Apple Silicon, recent NVIDIA/AMD cards). For the AI community, this demonstrates that in-browser inference is no longer a compromise; with well-tuned kernels, even complex models can achieve production-ready speeds while maintaining user privacy and zero latency.