Qwen 3.6 27B Impresses on Single Prompts but Fails at Agentic Tasks
Users find the 27B model makes critical errors every 4 turns in multi-step workflows.
Despite the buzz around Qwen 3.6 27B, a Reddit user (TokenRingAI) reports that the model is unsuited for agentic work—autonomous, multi-step tasks where AI must follow instructions and correct itself. On single prompts, the 27B model excels at generating impressive demo HTML pages and producing longer content than its predecessor, the 3.5 series. At 8-bit and 16-bit quantization on llama.cpp (nightly build, RTX 6000), it consistently underperforms when asked to maintain context over multiple turns. The user notes that “every 4 turns or so it does something completely braindead,” ignoring directions and making continuous mistakes.
This stands in stark contrast to the Qwen 3.5 122B model, which the same user has run at 4‑bit and 5‑bit quantization with far better reliability for agentic workloads. While 122B is slower and more resource-intensive, its ability to follow directions and avoid catastrophic errors makes it the preferred choice for practical agent implementation. The finding underscores a growing concern in the LLM community: benchmark performance on isolated prompts does not guarantee robustness in autonomous agent loops, where reliable instruction-following and self-correction are critical.
- Qwen 3.6 27B at 8-bit/16-bit on llama.cpp excels at single-prompt tasks like generating demo HTML pages.
- It fails in agentic work, making critical errors every 4 turns and frequently ignoring instructions.
- Users report better multi-turn reliability with Qwen 3.5 122B at 5-bit quantization, despite higher resource usage.
Why It Matters
Shows that even capable models can fail in agent deployments, underscoring the need for real-world multi-turn testing.