LLaMA quantization study reveals silent reasoning failures accuracy can't catch
Quantized LLMs get answers right for wrong reasons—and standard metrics won't spot it.
A new study published on arXiv presents a taxonomy of silent reasoning failures in quantized large language models, analyzing 30,000 chain-of-thought outputs from five models (3B–14B parameters) at three precisions (FP32, FP16, NF4). The researchers identified six failure categories, with two standout phenomena: Hollow Convergence—where a model reaches the correct answer through incomplete or unverifiable reasoning—and Shortcut Collapse, where models fall back on pattern-matching instead of logical steps. Inter-rater agreement was strong (Cohen's κ=0.906).
Key findings show these failures are size- and benchmark-dependent. Under NF4 quantization, Hollow Convergence drops sharply for smaller models (<12B) but stays stable for larger ones. Shortcut Collapse skyrockets from 44% to 78% of wrong-answer failures in LLaMA 3.2-3B, while Confidence Snowballing nearly vanishes—a qualitative shift invisible to accuracy metrics. Critically, Hollow Convergence cannot be reliably detected from surface text features (F1=0.53), meaning standard eval pipelines are blind to this failure mode. The study highlights a dangerous gap: efficiency-focused quantization can silently break reasoning integrity without impacting accuracy scores.
- Hollow Convergence (correct answer from flawed reasoning) increases significantly under NF4 for models under 12B parameters, but remains stable for larger models.
- Shortcut Collapse in LLaMA 3.2-3B jumps from 44% to 78% of wrong-answer failures under NF4, while Confidence Snowballing drops from 15.8% to near zero.
- Surface-level text features cannot reliably detect Hollow Convergence (best F1=0.53), making existing evaluation pipelines inadequate for quantized models.
Why It Matters
Quantization for performance may introduce invisible reasoning flaws that accuracy benchmarks cannot catch—key for safety in production.