PrismML's Bonsai 27B shrinks to 3.9GB, runs locally on iPhone

PrismML has released Bonsai 27B, a 1-bit quantized version of the Qwen3.6-27B model that fits entirely on an iPhone. By applying aggressive binary quantization—each weight reduced to a single sign bit plus one FP16 scale per 128 weights—the model size plunges from ~54GB to just 3.9GB. This allows the 27-billion parameter model to run locally on devices like the iPhone 15 Pro Max (8GB RAM) via the Atomic Chat app. Remarkably, even the embeddings, attention projections, MLP layers, and LM head are binarized, avoiding the common practice of keeping some layers at higher precision. The result is a ~1.125 bits-per-weight average with no high-precision escape hatches.

Benchmark performance remains strong: Bonsai 27B averages 76.1 across 15 benchmarks versus 85.1 for the FP16 original, a 89.5% retention. Math tasks hold up especially well at 91.7%, while knowledge and reasoning drop to 73.4 (vs. 83.2)—expected trade-offs from extreme quantization. Memory is manageable at ~5.2GB with 4K context and ~6.8GB at 100K context using 4-bit KV cache. This breakthrough makes it feasible to run a frontier-scale LLM entirely on-device without internet, opening doors for privacy-sensitive, always-available AI assistants on phones.