Qwen 3.6 35B beats 27B in agentic tasks — KV cache is the hidden bottleneck
Unquantized KV cache let the 35B IQ4NXL one-shot what the 27B Q5KXL couldn't debug in hours.
In a hands-on agentic workflow test (building an MCP subgraph with 11 tools in Rivet, integrating with OpenWebUI and llama.cpp), a developer discovered that Qwen 3.6 35B in IQ4NXL quantization with full (uncompressed) KV cache outperformed the widely-praised 27B Q5KXL at KV Q8/8. The 27B, historically considered smarter and a daily driver, failed repeatedly — causing context overflows, intelligence drops, and hours of debugging — while the 35B nearly one-shot the solution. The developer attributes this not just to model size, but to the critical role of KV cache precision: even a modest KV quantization (Q8/8) on the 27B degraded its ability to handle 30K+ token agentic contexts with memory injection and trimming.
The 35B IQ4NXL run without KV compression maintained stable reasoning across long sessions, while the 27B required switching to Q4KXL at KV Q4/4 to avoid slowdowns, risking forgotten routines in session summaries. The developer notes that the 35B does slow down under very high context, but it retains intelligence where the 27B fails. For agentic work — where context integrity is paramount — KV cache quality can make or break a model's utility. This real-world test underscores that low-bit KV cache (even Q8) is not a free lunch, and that the right quantization strategy depends heavily on task type: smart models without sufficient KV precision can become liabilities in autonomous workflows.
- Qwen 3.6 35B (IQ4NXL, full KV cache) solved a complex agentic task in one attempt that the 27B Q5KXL (KV Q8/8) failed after days.
- KV cache compression (even Q8/8) caused context overflow and intelligence labotomy in the 27B during long-context agentic workflows.
- The 35B runs slower at high context but avoids forgotten routines; its unquantized KV cache was the decisive factor over model intelligence.
Why It Matters
For autonomous agents and long-context tasks, KV cache precision is as critical as model size — choose quant wisely.