Qwen3.8-27B shows significant knowledge drop vs Qwen3.6 in benchmarks
Qwen3.8-27B stumbles on obscure facts that Qwen3.6 nailed, irking power users.
After days of stress-testing Alibaba's Qwen3.8-27B across personal workflows, a Reddit user noticed a troubling regression: the model consistently failed a private set of obscure trivia questions that Qwen3.6 answered without issue. This held across all quantization levels and sampling settings tested. The poster then checked offline knowledge benchmarks (no tool calls) and found the data aligns—Qwen3.8-27B is significantly weaker than Qwen3.6 at recalling random facts and resists hallucinating less effectively, despite strong performance in coding and reasoning tasks.
The user's conclusion is pragmatic: knowledge retrieval from raw model weights is becoming an unwise strategy, especially for airgapped or fully offline deployments. They recommend relying on tool calls—like MCP servers or external knowledge bases—for factual lookup, and note that Qwen3.8-27B still shines in code generation and agentic tasks. While the drop may not matter for most users who already use retrieval-augmented pipelines, it's a critical warning for those who expect an all-in-one model to be a reliable encyclopedia.
- Qwen3.8-27B fails obscure trivia that Qwen3.6 answered reliably across all quantization levels
- Offline knowledge benchmarks confirm a significant regression vs Qwen3.6
- Coding and tool-use remain strong; the author recommends MCP/tool calls for factual retrieval
Why It Matters
Alibaba's update exposes a trade-off between reasoning and knowledge—offline users must now lean on external retrieval instead of model memory.