PyTorch fixes Blackwell FP8 compilation crash with swizzled scale fallback
Blockwise MXFP8/NVFP4 compile tests on Blackwell now pass without assertion errors...
PyTorch has pushed a critical fix for TorchInductor, addressing an assertion crash when compiling blockwise MXFP8 and NVFP4 operations on NVIDIA’s new Blackwell GPUs. The issue arose in the `_scaled_mm_v2` lowering path, which had no template or extern implementation for non-trivial swizzle patterns in the scale layout. This caused compilation to fail during testing of low-precision float formats that are increasingly important for high-performance AI workloads. The patch was authored with assistance from Claude Opus 4.8 and merged into the trunk after review by drisspg.
The solution is elegant: for swizzled scale layouts (non-trivial patterns), the compiler now defers to the eager `_scaled_mm_v2` operator rather than trying to compile it through TorchInductor. The trivial-swizzle path remains entirely unchanged, preserving optimal performance for common cases. This PR (pytorch/pytorch#186384) references commit 9eb8bcb2f55 and ensures that users compiling models with blockwise FP8 formats on Blackwell GPUs won’t hit assertion errors during code generation. The fix is essential for researchers and engineers working with the latest NVIDIA hardware and PyTorch’s eager compilation stack.
- Fixed TorchInductor crash for non-trivial swizzle patterns in _scaled_mm_v2 on Blackwell GPUs
- Defer to eager _scaled_mm_v2 for swizzled scale layouts; trivial path unchanged
- Commit authored with Claude Opus 4.8, ensuring stable compilation of MXFP8/NVFP4 operations
Why It Matters
Enables stable PyTorch compilation of next-gen FP8 formats on Blackwell, critical for AI model performance