Developer Tools

PyTorch fixes fake mode detection for tensor subclass graphs

Subclass outputs now share the same FakeTensorMode, fixing dynamic shapes.

Deep Dive

A recent commit to PyTorch's `standalone_compile` function fixes a subtle bug in fake mode detection when the compiled graph returns tensor subclass objects. The function `standalone_compile(dynamic_shapes="from_graph")` is designed to recover the `FakeTensorMode` from graph output metadata so that Inductor can preserve dynamic shape state from the captured graph. However, the original implementation only accepted output example values that were directly `FakeTensor` instances. When the graph returned a traceable tensor subclass—whose inner tensors were `FakeTensors`—the direct `isinstance` check missed the existing fake mode. As a result, `standalone_compile` fell back to a fresh `FakeTensorMode` with a fresh `ShapeEnv`, losing important dynamic shape information.

The fix replaces the direct `FakeTensor` check with a call to `maybe_get_fake_mode`, a helper that already knows how to recurse through traceable wrapper subclasses and related wrappers. This allows the `from_graph` path to reuse the same fake mode for both ordinary `FakeTensor` outputs and tensor subclass outputs, keeping the change local to fake mode detection without adding a tensor-subclass special case. The commit, authored by jansel and linked to issue #151945, includes microbenchmarks showing the overhead: the new detection runs in about 0.2 microseconds for direct `FakeTensor` (vs 0.086 previously) and 1.35 microseconds for `TwoTensor` subclasses (vs 0.112). While slightly slower, the correctness gain is crucial for users relying on dynamic shapes with custom tensor subclasses.

Key Points
  • Fix replaces `isinstance` check with `maybe_get_fake_mode` to handle tensor subclass wrappers.
  • Preserves dynamic shape state from captured graphs for subclass outputs, fixing issue #151945.
  • New detection adds ~0.1µs overhead for raw tensors and ~1.2µs for TwoTensor subclasses.

Why It Matters

Ensures dynamic shape preservation for custom tensor subclasses in PyTorch compilation.

📬 Get the top 10 AI stories daily