Quantized LLMs show 244x eigenvalue spike — new monitor proposed

A new paper from Alekberli and Karimov tackles a practical problem for deployed LLMs: how does weight quantization (e.g., 4-bit) affect the statistical geometry of a model? They study the spectral perturbation of the empirical Fisher Information Matrix (FIM) under two structured perturbations: input distribution shift and finite-precision parameter quantization. The FIM's largest eigenvalue λ_max serves as a curvature measure of the loss landscape. Using Weyl's inequality and a local curvature-monotonicity hypothesis, they prove (Theorem 4.3) that under quantization noise, λ_max is lower-bounded by its unperturbed value plus a third-order remainder, and under a mild genericity condition strictly exceeds it at leading order.

The authors propose using σ_t = λ_max(F_t)/λ_base as a runtime monitoring statistic for deployed language models. In experiments on 12 models with 1,080 trajectories, they found the calibration threshold for this statistic was approximately 244 times larger on a 4-bit quantized model compared to a full-precision baseline — a striking empirical observation that aligns with their theoretical predictions. The paper openly states the closed-form prediction of quantization inflation magnitude as an unsolved problem. This work bridges theoretical spectral analysis with practical LLM monitoring, potentially enabling early detection of performance degradation in quantized models during deployment.