trunk/576abe9c35ddfcad8abb1d2dd5a50b160e840c8c: Codegen RuntimeWrapper orchestration into single function (#181271)
PyTorch's new codegen cuts runtime overhead by up to 3.2x.
PyTorch's latest pull request (#181271) introduces a significant performance optimization by consolidating several runtime wrapper functions—including `_RuntimeCompiledFnInvoker.run`, `_RuntimeForwardEpilogue.capture_orig_inputs`, `increment_mutation_versions`, and `finalize`—into a single codegen'd function. By resolving all branches at compile time, the generated function eliminates method dispatch overhead, inlining operations like dict comprehension for input capture, conditional logic for mutation versioning, and trace joint branch handling for compiled invocations.
Benchmarks show speedups ranging from 2.1x to 3.2x across various configurations, with the most complex case (5 aliases, 3 mutations, 20 inputs) dropping from 0.93 us to 0.32 us per call. This optimization directly benefits PyTorch's compilation pipeline, reducing runtime overhead for both inference and training workloads. The change is approved and merged, marking a notable step forward in PyTorch's ongoing performance improvements.
- Consolidates 4 runtime wrapper functions into a single codegen'd function
- Achieves 2.1x to 3.2x speedup across various alias/mutation scenarios
- Inlines input capture, mutation tracking, output validation, and grad disabling
Why It Matters
Reduces PyTorch's runtime overhead, accelerating compiled model execution for developers.