PyTorch's new PR adds premul_sum and min/max gradients to all_reduce

PyTorch has merged PR #190942, 'Add more ops support for functional all_reduce,' which significantly expands the capabilities of its collective communication API. The change, co-authored with Claude Opus 4.6, addresses longstanding gaps in the functional all_reduce implementation. The first major enhancement is premul_sum support, allowing a multiplication factor to be applied to tensors before the reduction operation completes. This makes the schema's reduce_op argument more flexible—now typed as Any—so callers can pass a ReduceOp object directly with a supplementary scalar factor, enabling operations like scaled summation in a single collective step.

The second improvement adds backward support for min and max operations. Previously, gradients for these reduce ops were not implemented, forcing users to work around the limitation. The new approach compares the output value with each input locally to route gradients accurately, completing gradient propagation for the full set of reduction operations. The PR also parameterizes all existing all_reduce tests to cover these changes, ensuring the new functionality is robust across different configurations. For distributed training, this means more efficient stateful optimizations (like gradient clipping or scaling) and smoother auto-differentiation through collective ops—a practical win for high-performance PyTorch workloads.