Qwen3.6 35B-A3B on CPU beats GPU with Q8_0 quantization for coding task
Switching from Q4_K_M GPU to Q8_0 CPU unlocked surprising code generation quality.
Deep Dive
A Reddit user reported that a model performed much better when switched from Q4_K_M on GPU to Q8_0 on CPU, calling the slowdown worth it. After using "plan mode" and then instructing it to implement without changes, they noted the model "punches far, far above its weight."
Key Points
- Model used: Qwen3.6 35B-A3B with 3B active parameters, 35B total
- Q8_0 (8-bit) on CPU produced far better code than Q4_K_M (4-bit) on GPU
- Single prompt generated a flight simulator with mountains, clouds, procedural terrain in one HTML file
Why It Matters
For creative coding tasks, higher precision quantization can be more important than inference speed, even on CPU.