Developer Tools

trunk/7a17fc503a95b6e426dec2ab27991fe7a5e0c628: [dynamo] Honor patched forward after in-graph setattr (#177521)

A subtle bug in PyTorch's Dynamo compiler was causing patched model methods to be ignored during graph tracing.

Deep Dive

The PyTorch team has resolved a critical bug in its TorchDynamo just-in-time (JIT) compiler, identified as issue #167400. The problem stemmed from how `UnspecializedNNModuleVariable.call_function()` handled module dispatch. When a neural network module's `__dict__` did not contain a `forward` method, the compiler would short-circuit directly to the type-level `forward`. However, if a user patched the `forward` method using an in-graph `setattr` operation—a common technique for dynamic model modification—this change was only tracked as a pending side effect during the tracing phase. Consequently, the compiler's fast path would skip the user's patched method entirely, leading to incorrect execution.

The fix, proposed by contributor @codex and reviewed by @bobrenjc93, modifies the dispatch logic to consult `has_key_in_generic_dict()` before taking the fast path. This ensures the compiler checks for pending mutations. Furthermore, `call_method()` now uses the same pending-mutation-aware lookup, making instance-level method patches visible during tracing. This approach is deemed the correct long-term solution because it preserves the performance-optimized fast path for the vast majority of unpatched modules while aligning unspecialized module behavior with Dynamo's established model for handling traced attribute side effects. A regression test has been added to verify that patching a module's `forward` method inside a compiled `fullgraph` function now works correctly.

Key Points
  • Fixes a Dynamo compiler bug (issue #167400) where in-graph `setattr` modifications to a module's `forward` method were ignored.
  • Root cause was a fast-path optimization that bypassed pending side-effect tracking for method lookups.
  • Solution ensures consistency with Dynamo's side-effect model, allowing dynamic model patching without breaking the common-case performance optimization.

Why It Matters

This fix is crucial for researchers and engineers who dynamically modify PyTorch models, ensuring their changes are correctly compiled and executed by TorchDynamo.