Qwen3.5-9B Quantization Comparison

A detailed community analysis has performed a rigorous quantization sweep of Alibaba's Qwen3.5-9B model, comparing 46 different GGUF files to establish a data-driven selection guide. The evaluation uses KL Divergence (KLD) to measure how much a quantized model's probability distribution drifts from the original BF16 baseline, with lower scores indicating greater faithfulness. Perplexity (PPL) measures total prediction error. The results reveal significant performance differences between quantizers like bartowski, unsloth, and lmstudio, even for files with the same nominal quantization level (e.g., Q4_K_M).

For users constrained by VRAM, bartowski's IQ4_XS quant (4.93 GiB, KLD 0.0127) is the top recommendation, offering the best performance without dropping below Q4 precision. The analysis shows that Q2 and IQ2 quants are measurably worse, and repetition loops observed in text generation align with higher KLD scores. The full ranking, visualized in a Hugging Face Space, allows developers to pick the optimal file for their specific hardware and accuracy needs, moving beyond guesswork to informed selection.