PrismML's Bonsai 27B fits a 27B model in 10GB, but quality trails Q4

PrismML has launched Bonsai 27B, a ternary (1,0,-1) quantized version of Qwen3.6 27B, aiming to bring powerful models to memory-constrained hardware. Early tester Tim from AnythingLLM reports it uses only ~10GB of VRAM at 32K context on an M4 Pro, a dramatic reduction from the ~54GB typically required for the full-precision model. The model supports 256K context windows and multimodal input, making it practical for on-device agent workflows and computer-use applications like OpenComputer. However, initial community testing shows Bonsai 27B does not match the quality of standard Q4_K_XL quantization—it hallucinates more, struggles with tool-calling loops, and generally lags behind in reasoning benchmarks. The original title claiming “near fp16 precision” was retracted by the author, who clarified it is “clearly worse than Q4.” Despite this, Bonsai 27B is a significant step for ternary models at scale: it offers far better performance than a 2-bit quant of the same size, and for many users, the ability to run a 27B model locally on a single consumer GPU or laptop is transformative. The release includes GGUF and MLX variants, with future support for dFlash and MTP hinted. While not a replacement for full-precision or high-quality quants, Bonsai 27B unlocks new possibilities for local AI agents that previously required cloud GPUs or expensive hardware.