Developer Tools

PyTorch fixes silent CPU numerical bug in inductor's outer-loop fusion with tiling

A sneaky buffer reuse bug caused silent wrong results in PyTorch's CPU code generator.

Deep Dive

PyTorch has merged a critical bug fix (PR #185855) in its inductor CPU backend that addresses a silent correctness issue in outer-loop fusion with tiled loops. The bug occurred when the compiler fused outer loops and replaced intermediate global pointwise buffers with function-local buffers. These local buffers were sized based on non-fused inner dimensions only, assuming each fused outer loop iteration processed exactly one outer element. However, when vectorization tiled one of those fused outer loops, a single local buffer slot was incorrectly reused across multiple outer elements within the same loop iteration. This led to silent wrong numerical results—for example, a reduction computing row-specific means correctly, but a later pointwise phase reading stale values left by a different row.

The fix, authored by jansel, conservatively falls back to normal code generation for the local-buffer outer-loop-fusion path whenever any fused outer loop is tiled. This avoids the unsafe aliasing case while keeping the optimization active for untiled outer loops, which remain safe. The alternative—rewriting the local-buffer indexing to account for tiled outer extents—was deemed too invasive. Validation shows the bug caused absolute errors up to 0.21, which the fix reduces to ~1e-7. Benchmarking on the reproducer (single-threaded, 2000 calls after warmup) shows median runtime of ~20.7 µs, essentially unchanged from the buggy version (~21.1 µs). The fix also passes existing local-buffer fusion tests plus the new regression test. This patch is especially important for users relying on PyTorch's inductor compiler for CPU inference or training, where silent numerical bugs can undermine model correctness without obvious signs.

Key Points
  • Bug in PyTorch inductor's CPU outer-loop fusion caused silent numerical errors up to 0.21 when vectorization tiled fused loops.
  • Fix falls back to normal codegen for local-buffer fusion when any fused outer loop is tiled, preserving optimization for untiled cases.
  • Performance impact is negligible (~0.3 µs median slowdown on the reproducer), while numerical error drops to ~1e-7.

Why It Matters

Eliminates silent wrong results in PyTorch's CPU compiler, crucial for reliable model inference and training.

📬 Get the top 10 AI stories daily