Adaptive Kaczmarz on 96 Cores: Threshold-Greedy Fails, Live Reads Win
Greedy Kaczmarz diverges immediately on high-core machines – here’s how to fix it.
Evan Coleman presents the first systematic study of adaptive row selection (residual-weighted and greedy) in randomized Kaczmarz under asynchronous execution on shared-memory machines. Using 339 runs on a 96-core node with realized delays, the work directly addresses the challenge of sampling from a residual that lock-free workers are concurrently modifying with stale data. The paper validates its implementation against published sequential results and outlines a distributed two-level sampler motivated by these findings.
Four actionable findings emerge: (i) stability is governed by a boundary ℓ*(T) between sampling aggressiveness and thread count – below it, more aggressive sampling is strictly better, so practitioners should tune just inside the cliff. (ii) Threshold-greedy selection (the standard accelerated rule) is unstable at high thread counts and diverges almost immediately. (iii) Under-relaxation buys back the cliff at a predictable cost, providing a usable safety knob. (iv) Consistent-snapshot reads admit a rare, scheduling-dependent divergence that live (inconsistent) reads never exhibited, making inconsistent reads the cheaper and safer default. The results optimize large sparse least-squares and tomographic reconstruction workflows.
- Threshold-greedy Kaczmarz (standard accelerated rule) becomes unstable and diverges at high thread counts on shared-memory systems.
- A stability boundary ℓ*(T) defines the safe region for sampling aggressiveness vs. thread count; tuning to just inside the cliff maximizes performance.
- Inconsistent (live) reads are cheaper, safer, and avoid a rare divergence pattern seen with consistent-snapshot reads.
Why It Matters
Provides concrete tuning guidelines for parallelizing sparse linear solvers on multi-core hardware, critical for large-scale tomography and optimization.