Some Matrix Multiplication Engines Are Not As Accurate As We Thought

Specialized AI chips in GPUs, like NVIDIA's Tensor Cores, use a design trick for speed: they perform calculations with lower internal precision than the official FP32 standard suggests. This reduction in the 'accumulator' can unexpectedly degrade the accuracy of AI models. While current hardware is fixed, software can be adjusted. The finding is crucial for developers seeking maximum performance and for future hardware designers prioritizing both speed and reliability.