PyTorch's isFwGradDefined overload cuts op overhead up to 27% in microbenchmarks

PyTorch's latest commit (#191813) targets a subtle performance bottleneck in the autograd engine. isFwGradDefined, a function that checks whether a forward gradient is defined, is called extensively in torch/csrc/autograd/generated/VariableType*.cpp. The original implementation only accepted optional<Tensor>, which forced an unnecessary optional constructor call and a later has_value() check on every invocation. The new overload accepts a plain Tensor, eliminating both costs.

Using a Codex-generated microbenchmark that runs torch.sin, torch.add, and torch.addcmul with 1, 2, and 3 empty tensors, the author measured consistent improvements: sin 12.12%, add 14.95%, addcmul 11.97%, and add_inplace 27.07%. While these numbers are compelling, the commit explicitly cautions that this is not representative of real-world performance. Still, for high-frequency operations in PyTorch, shaving even 10-20% off a hot path can reduce overhead at scale, especially for deep learning workloads that invoke thousands of tiny tensor operations per step.