Developer Tools

PyTorch Dynamo now supports mark_dirty in autograd Function tracing

Fixes a gradient preservation bug when dirty tensors are returned through input aliases.

Deep Dive

PyTorch's latest commit (45a8b48) introduces support for the mark_dirty context method in Dynamo's autograd Function HOP (Higher Order Operator). Previously, when a custom autograd.Function marked an output as dirty via ctx.mark_dirty, Dynamo's tracing would drop that semantic information, causing incorrect gradient computations if the dirty tensor was later observed through the original input alias. This PR fixes issue #184113 by ensuring Dynamo records the mark_dirty calls during tracing and faithfully replays them in the synthesized HOP wrapper.

The solution is carefully scoped: only dirty tensors that are also returned and then observed through an alias of the original input trigger gradient preservation. If a user marks an input as dirty but does not return it, Dynamo still graph breaks rather than silently producing incorrect eager semantics. This maintains safety while extending Dynamo's ability to compile a wider range of custom autograd functions. The change was authored by Codex (an AI tool) and reviewed by zou3519, with test plans covering dynamo autograd function scenarios. This is a notable step in making PyTorch's dynamic compilation more robust for advanced gradient use cases.

Key Points
  • Dynamo now records ctx.mark_dirty during autograd.Function tracing and replays it in the synthesized HOP wrapper.
  • Fixes gradient preservation for dirty returned tensors observed through original input aliases (issue #184113).
  • Invalid dirty inputs that are not returned still cause a graph break instead of silent incorrect compilation.

Why It Matters

Ensures correct gradient computation for custom autograd functions, improving PyTorch's dynamic compilation reliability.