Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models

A team of researchers from Florida International University and other institutions has published a groundbreaking paper on arXiv that diagnoses a fundamental, hardware-level source of unpredictability in Large Language Models (LLMs). Their analysis reveals that the finite numerical precision of floating-point representations—the way computers approximate real numbers—causes rounding errors that propagate unpredictably through the Transformer architecture's computation layers. The study identifies a critical 'avalanche effect' in early layers, where minuscule perturbations can be rapidly amplified or completely attenuated, leading to binary, divergent outcomes from nearly identical starting points.

Beyond isolated errors, the research demonstrates that LLMs exhibit universal, scale-dependent chaotic behavior characterized by three distinct regimes. In the stable regime, perturbations below a certain threshold vanish, resulting in consistent outputs. In the chaotic regime, rounding errors dominate and drive outputs to diverge unpredictably. Finally, in the signal-dominated regime, genuine variations in the input override the numerical noise. The team validated these findings across multiple datasets and model architectures, providing a rigorous mathematical framework for a phenomenon many developers have observed anecdotally: that LLMs can be surprisingly non-deterministic.