PyTorch Just Rolled Back Its Latest Optimization — What That Means for Your Models
A new performance optimization for variance computations gets automatically rolled back.
Deep Dive
PyTorch has reverted a commit that introduced an L2-aware two-pass variance heuristic from PR #183661. The revert was automatically triggered by PyTorch's autorevert system; to prevent future automatic reversions, add the tag "autorevert: disable".
Key Points
- PyTorch automatically reverted commit 76f2fe430c from PR #183661 due to an error.
- The reverted optimization was an L2-aware two-pass variance heuristic for improved accuracy and speed.
- Developers on the latest trunk will need to use alternative variance computations until a fix is re-applied.
Why It Matters
A performance-critical optimization for variance operations gets rolled back, affecting training stability for many PyTorch models.