Bonsai-27B 2-bit scores 7.9% on Terminal-Bench, lags behind Qwen 9B
8GB VRAM fits both models, but 1-bit Bonsai fails in agentic loops with 14k-token rambles.
In a recent Reddit post, user Creative-Regular6799 published detailed benchmark results for two extreme quantization variants of the Bonsai model family: Ternary-Bonsai-27B (2-bit) and Bonsai-27B (1-bit). The tests were run on an RTX 5070 Laptop 8GB with an i9-14900HX and 32GB RAM using a llama.cpp fork from PrismML (stock llama.cpp cannot load 2-bit kernels). The benchmark harness was little-coder via harbor adapter, covering all 89 tasks of Terminal-Bench 2.0 with a single attempt (k=1), 40-turn cap, and temperature 0.2.
Results show that Ternary-Bonsai-27B at 2-bit achieved only 7.9% accuracy, compared to Qwen3.5-9B's 9.2% and Qwen3.6-35B-A3B's 24.3% (both from k=5 runs). While the 2-bit model did fit entirely in 8GB VRAM and produced zero parse errors, its accuracy is actually lower than the much smaller Qwen 9B running at normal Q4 quantization. The 1-bit Bonsai model was unusable in an agentic loop: it generated a single 14,000+ token completion on the first task, never emitting a stop token, and rambled until exhausting the 32k context window. The tester aborted after observing self-validation tics that snowballed into non-termination. On simple prompts like math or prime checks it works fine, but in any agentic harness it fails completely.
- Ternary-Bonsai-27B at 2-bit scores 7.9% on Terminal-Bench 2.0, below Qwen3.5-9B's 9.2%
- 1-bit Bonsai-27B fails in agentic loops: 14,000+ token ramble with no stop, exhausting 32k context
- Both models fit in 8GB VRAM, but 2-bit accuracy trade-off makes a smaller dense Q4 model more practical
Why It Matters
Extreme quantization (2/1-bit) enables large models on low VRAM, but real agentic accuracy still favors smaller dense models.