PyTorch's NVGEMM backend gets automatic epilogue fusion for faster GPU kernels

PyTorch's Inductor compiler just landed a major performance upgrade for NVIDIA GPUs: epilogue fusion in the NVGEMM backend. Epilogue fusion combines post-GEMM operations (— like activations or simple arithmetic —) into the same kernel, eliminating extra memory round-trips. The commit, authored with help from Claude, adds support for relu, sigmoid, tanh, exp, add, sub, mul, div, and dtype casts via CUTLASS's EpilogueArguments. The scheduler’s fusion loop recognizes NVUniversalGemmCaller and dynamically benchmarks fused versus unfused EFC kernels, picking the clear winner for each shape. If NVGEMM can’t fuse a particular pattern (e.g., unusual combinations), the code falls back to Triton’s own epilogue fusion attempts.

This is a classic compiler-level optimization: instead of forcing users to manually fuse activations with their matmuls, PyTorch now does it automatically and adaptively. The PR includes test coverage via test_nv_universal_gemm.py and test_max_autotune.py. Practically, this means faster training and inference for any model that uses common activation functions after linear layers, especially in transformer architectures. Combined with PyTorch's existing Triton support, this gives NV users a robust, best-of-both-worlds approach to kernel fusion.