Developer Tools

PyTorch adds test for swizzled scales in scaled_mm compile

Inductor now handles SWIZZLE_32_4_4 scales for MXFP4/8 on B200.

Deep Dive

PyTorch has merged a test confirming that the Inductor compiler can now handle swizzled scales in the _scaled_mm_v2 operation. Previously, the lowering logic asserted that all swizzles must be NO_SWIZZLE, which broke torch.compile for scaled_mm operations using MX (microscaling) and NVFP4 recipes—these require NVIDIA's SWIZZLE_32_4_4 pattern on Blackwell B200 GPUs. The fallback fix landed in PR #186384; the new PR (#186948) adds the missing regression test to ensure the fix works.

The test compiles a blockwise MXFP8 scaled_mm with swizzled scales, compares the compiled result against eager execution, and asserts that the generated code calls the ATen kernel. It is B200-only (PLATFORM_SUPPORTS_MX_GEMM), matching the other blockwise tests. Authored with an AI assistant, this addition reinforces PyTorch's support for low-precision matrix multiplication on NVIDIA's latest architecture. For developers working with MXFP4/FP8 models, this means they can now fully leverage torch.compile to optimize performance without encountering compilation errors due to scale swizzle requirements.

Key Points
  • Added regression test for PyTorch Inductor's _scaled_mm_v2 with swizzled scales (SWIZZLE_32_4_4)
  • Test compiles blockwise MXFP8 scaled_mm on B200 hardware and validates against eager execution
  • Follows fallback fix #186384 that removed the NO_SWIZZLE assertion blocking MX/NVFP4 recipes

Why It Matters

Ensures PyTorch can compile MXFP4/FP8 models with swizzled scales on B200 for production workloads.

📬 Get the top 10 AI stories daily