trunk/69daaed349063a323387f20484466fc93bb32b66: Formalize out= operators and custom operators definition (#180851)
PyTorch 2.10's new API standardizes memory-efficient operations, breaking BC for cleaner operator design.
The PyTorch team at Meta has merged a significant pull request (#180851) that formalizes the definition and behavior of 'out=' operators within the framework's operator system. This change renames the internal 'Tag::out_variant' to simply 'Tag::out', a move made possible because the older API wasn't widely used in production, minimizing backward compatibility (BC) concerns. More importantly, the update now automatically applies this 'out' tag to all native PyTorch operators that use the out= parameter pattern. The Python API (torch.library.define) now validates that any custom operator tagged as 'out' adheres to three strict schema constraints: all mutable arguments must be write-only buffers, the operator must return all mutable arguments in order, and those arguments must be keyword-only.
This formalization addresses a design gap. Previously, the framework lacked a clear, enforced contract for what constitutes a proper 'out' operator, even though torchgen (PyTorch's operator registration code generator) already imposed similar restrictions on native operators. The change breaks BC in one specific area: operators tagged with 'out' can no longer return 'None' or void, a behavior that was theoretically allowed but not used in practice. By establishing a clear, validated definition, PyTorch provides a more robust foundation for developers building custom, memory-efficient operations. This is crucial for performance-sensitive applications where avoiding unnecessary memory allocations by writing results directly to pre-allocated tensors is a common optimization technique.
- Renames internal 'Tag::out_variant' to 'Tag::out' for clarity, as the feature wasn't widely used, minimizing BC impact.
- Enforces three key schema rules via torch.library.define: write-only mutable buffers, ordered returns, and kwarg-only mutable args.
- Breaks BC by disallowing 'out' tagged operators from returning None/void, a theoretical feature not used in production.
Why It Matters
Provides a standardized, validated API for creating memory-efficient custom operators, improving performance and code reliability for ML engineers.