PyTorch tightens Inductor tolerance to fix sum reduction accuracy

PyTorch's inductor compiler, which accelerates model execution, recently received a numerical precision fix in PR #190335. The change addresses a post-merge review comment from a prior PR that scaled both relative tolerance (rtol) and absolute tolerance (atol) per dtype for cooperative reduction checks. That approach created a problem: for the sum reduction, which involves large-magnitude outputs (~1e3) with relative accumulation error (~1e-4), the rtol * |expected| budget (~1.4) easily covered the error, but the absolute error (~1e-1) never reached the loosened atol. However, the same tolerance parametrization was also used for small-magnitude outputs like softmax entries (~1e-6) and mean over 1M elements (~1e-3), where a loosened atol (e.g., 1e-2) made the numeric checks nearly vacuous.

The fix keeps atol tight at 1e-6 and scales only rtol per dtype, preserving sensitivity for small outputs while still covering the relative error in sum. The change is test-only and verified through CI on CUDA builds. For practitioners using PyTorch's inductor for training or inference, this ensures that reduction operations like sum, softmax, and mean maintain accurate numerical comparisons without false passes or silent failures.