PyTorch fixes CUDA crash when only attention mask needs gradients
Memory-efficient attention backward now saves logsumexp for mask gradient computation.
PyTorch has patched a CUDA illegal memory access bug in `torch.nn.functional.scaled_dot_product_attention` (SDPA) that occurred when the float attention mask was the only input requiring gradients. The issue manifested when using the memory-efficient SDPA backend (EFFICIENT_ATTENTION). During the forward pass, the top-level SDPA wrapper only requested the `logsumexp` tensor from fused forward when `query`, `key`, or `value` required gradients. If only `attn_mask` required gradients, `logsumexp` was skipped, causing backward to receive an empty tensor and the CUDA kernel to read through a null pointer – resulting in an illegal memory access error.
The fix updates `aten/src/ATen/native/transformers/attention.cpp` so that `should_compute_logsumexp` also considers whether the attention mask requires gradients. For the memory-efficient branch, the check occurs after mask preprocessing (padded/expanded masks) to preserve autograd requirements. A CUDA regression test was added in `test/test_transformers.py` that forces EFFICIENT_ATTENTION, sets `requires_grad` only on the mask, and verifies the mask gradient matches the math backend. This ensures that users who pass learnable attention masks (e.g., for learned positional biases or gating) no longer encounter silent crashes during backward.
- Bug caused CUDA illegal memory access in `scaled_dot_product_attention` when only `attn_mask` required gradients and using memory-efficient backend.
- Root cause: `logsumexp` tensor was not saved during forward when query/key/value lacked gradients, even though backward needed it for mask gradient.
- Fix: Updated `should_compute_logsumexp` logic in attention.cpp to include mask gradient requirement, plus added regression test.
Why It Matters
Ensures reliable training of attention-based models with learnable attention masks without CUDA crashes.