unsloth - MiniMax-M2.7-GGUF in BROKEN (UD-Q4_K_XL) --> avoid usage
A popular quantizer's 'UD-Q4_K_XL' model returns NaN perplexity scores, indicating a broken, unchecked release.
A critical post from a member of the open-source AI community has called out Unsloth, a well-known provider of quantized AI models, for releasing a broken version of the MiniMax-M2.7 model. The specific file, 'MiniMax-M2.7-UD-Q4_K_XL.gguf', was found to be fundamentally flawed when tested using the standard `llama-perplexity` tool, returning NaN (Not a Number) values. This indicates catastrophic numerical errors within the quantized model, rendering it unusable. The issue is attributed to a rushed publishing process that skipped essential quality assurance (QA) checks, such as validating quant blocks and measuring perplexity (PPL) and KL divergence (KLD) metrics before release.
In contrast, the user tested alternative quantizations of the same MiniMax-M2.7 model from other Hugging Face providers, aessedai and ubergarm, which performed correctly without NaN errors. The post argues this is part of a broader pattern where quantizers prioritize being 'first to market' to capture user demand over delivering reliable, validated models. It calls on Unsloth and similar providers to adopt the community's established QA practices, including transparently publishing PPL/KLD data and using validation flags like `--validate-quants` to prevent such failures. The incident underscores the technical risks for developers and researchers who depend on these community-shared model files for their projects.
- Unsloth's 'MiniMax-M2.7-UD-Q4_K_XL' GGUF file returns NaN in perplexity tests, proving it's a broken quantization.
- The error is blamed on a rushed release cycle lacking standard QA like PPL/KLD measurement or block validation.
- Alternative quants of the same model from aessedai (Q5_K_M) and ubergarm (IQ5_K) were tested and found to be functional.
Why It Matters
For professionals deploying local models, unreliable quantized files waste time, compute resources, and can derail project timelines, demanding more rigorous sourcing.