PyTorch fixes FlexAttention cache miss with improved torch._check handling
A 9-day-old bug in PyTorch's torch._check was breaking AOTAutograd caching for FlexAttention.
PyTorch's FlexAttention module suffered a subtle cache-busting bug introduced roughly nine days before the fix. The issue originated from a recent change (PR #183838) that added inline torch._checks with unbacked symints for better Dynamo compatibility. However, those checks used temporary lambda functions as error messages—closures that could not be pickled. When AOTAutograd or Inductor attempted to serialize the FX graph for cache-key computation (e.g., with fx_graph_cache or autograd_cache enabled), pickling failed and forced a cold compile every time, defeating the purpose of caching.
The fix, implemented in commit 723fb7187aa9 and merged as PR #188177, takes a two-pronged approach. First, the torch._dynamo/variables/torch.py variable handler now explicitly accepts str and None constants as torch._check* messages, matching eager-mode behavior. The dispatch was rewritten using match/case for clarity. Second, flex_attention.py replaced its nested local function error helpers (like block_mask_too_small_error) with module-level string constants, ensuring all messages are picklable. Validation tests confirm that compiled FlexAttention now properly saves and hits the AOTAutograd cache, eliminating the cache-miss warning and restoring performance for users running dynamic attention graphs.
- PR #188177 fixes a cache miss in FlexAttention caused by unpicklable lambda functions in torch._check error messages.
- The solution extends torch._check to accept string constants (matching eager behavior) and refactors FlexAttention to use module-level string messages.
- Validation shows compiled FlexAttention now correctly saves and hits the AOTAutograd cache, including with fx_graph_cache and autograd_cache enabled.
Why It Matters
Fixes cold-compilation penalty for PyTorch users relying on caching with compiled FlexAttention, important for large-scale model training.