PyTorch fixes fake mode detection for tensor subclass graphs
Subclass outputs now share the same FakeTensorMode, fixing dynamic shapes.
A recent commit to PyTorch's `standalone_compile` function fixes a subtle bug in fake mode detection when the compiled graph returns tensor subclass objects. The function `standalone_compile(dynamic_shapes="from_graph")` is designed to recover the `FakeTensorMode` from graph output metadata so that Inductor can preserve dynamic shape state from the captured graph. However, the original implementation only accepted output example values that were directly `FakeTensor` instances. When the graph returned a traceable tensor subclass—whose inner tensors were `FakeTensors`—the direct `isinstance` check missed the existing fake mode. As a result, `standalone_compile` fell back to a fresh `FakeTensorMode` with a fresh `ShapeEnv`, losing important dynamic shape information.
The fix replaces the direct `FakeTensor` check with a call to `maybe_get_fake_mode`, a helper that already knows how to recurse through traceable wrapper subclasses and related wrappers. This allows the `from_graph` path to reuse the same fake mode for both ordinary `FakeTensor` outputs and tensor subclass outputs, keeping the change local to fake mode detection without adding a tensor-subclass special case. The commit, authored by jansel and linked to issue #151945, includes microbenchmarks showing the overhead: the new detection runs in about 0.2 microseconds for direct `FakeTensor` (vs 0.086 previously) and 1.35 microseconds for `TwoTensor` subclasses (vs 0.112). While slightly slower, the correctness gain is crucial for users relying on dynamic shapes with custom tensor subclasses.
- Fix replaces `isinstance` check with `maybe_get_fake_mode` to handle tensor subclass wrappers.
- Preserves dynamic shape state from captured graphs for subclass outputs, fixing issue #151945.
- New detection adds ~0.1µs overhead for raw tensors and ~1.2µs for TwoTensor subclasses.
Why It Matters
Ensures dynamic shape preservation for custom tensor subclasses in PyTorch compilation.