FP4 Tensor Cores crack FP64 matrix math with base-13 trick
Researchers emulate FP64 DGEMM on FP4 tensor cores, beating FP8 implementations.
FP64 matrix multiplication (DGEMM) is essential for scientific computing but slow on GPUs. Tensor Cores only handle low precision, so researchers use Ozaki schemes to split numbers into limbs. Prior work used INT8 or FP8; this paper is the first to use FP4 (E2M1), which is theoretically twice as fast. The key insight: doubling any FP4 value yields an integer, and shifting that integer set by multiples of 13 covers all integers. This allows arbitrary integers to be represented as base-13 FP4 limbs, and since FP32 accumulators handle sums up to a certain magnitude, intermediate products stay error-free.
The authors implement both Ozaki scheme I and II on FP4 Tensor Cores of an RTX PRO 6000 Blackwell. They also show that the same principle lets INT8 Tensor Core GEMM be emulated bit-exactly on FP4. With kernel-level optimizations to maximize occupancy and instruction scheduling, their FP4 Ozaki II achieves measurable speedups over the theoretical advantage. At 16384³ problem size, their implementation actually outperforms an existing FP8-based Ozaki II, making FP4 the new best option for high-precision HPC workloads on consumer GPUs.
- Base-13 limb representation keeps FP4 intermediate sums error-free in FP32 accumulators
- Achieves bit-exact INT8 GEMM emulation on FP4 Tensor Cores
- Outperforms FP8-based Ozaki II at 16384³ on RTX PRO 6000 Blackwell
Why It Matters
Faster FP64 emulation via FP4 tensor cores could slash HPC costs and accelerate scientific simulations on mainstream GPUs.