NVIDIA Tensor Cores gain multiple double arithmetic via Ozaki-like scheme
A new method unlocks high-precision math on tensor cores without branching.
A new paper by Howard Chen and Jan Verschelde, accepted for the International Conference on Mathematical Software (ICMS 2026), presents a method to run multiple double arithmetic on NVIDIA Tensor Cores. Multiple double arithmetic represents high-precision numbers as an unevaluated sum of standard double-precision floats, but each operation requires renormalization—a process involving branching that conventional tensor cores cannot handle. The researchers bypass this limitation by adapting the Ozaki scheme, originally designed for quad-double arithmetic, to avoid branching entirely. This allows tensor cores on the NVIDIA Ampere A100 (released 2020) to perform 64-bit floating-point matrix operations at high precision.
The work opens up new possibilities for scientific computing on consumer and data-center GPUs, where tensor cores are abundant but previously limited to lower-precision matrix math. By enabling multiple double arithmetic—supporting quad-double (four doubles) or oct-double (eight doubles) precision—the technique can accelerate applications in numerical analysis, computational geometry, and physics simulations that demand more than double precision. The authors have released their software under the GPU GPL license on GitHub, making it freely available for the research community. This approach could also inspire similar hacks for other hardware accelerators with limited control flow support.
- Problem: Tensor cores lack branching for renormalization required by multiple double arithmetic.
- Solution: Adapted Ozaki scheme to avoid branching, enabling high-precision ops on Ampere A100 Tensor Cores.
- Impact: Open-source software (GPU GPL) brings high-precision matrix operations to NVIDIA GPUs for scientific computing.
Why It Matters
It democratizes high-precision numerical computing by enabling tensor cores, boosting scientific simulations on GPUs.