Developer Tools

PyTorch Inductor preserves pin_memory for tensor constructors

Pinned memory annotations now survive compiler optimizations in PyTorch's Inductor

Deep Dive

The PyTorch team has merged a critical fix (PR #183977) for its Inductor JIT compiler, ensuring that pinned memory (pin_memory=True) annotations on tensor constructors are preserved during compilation. Previously, when using Inductor to compile operations like torch.tensor or torch.rand with pin_memory=True, the compiler could replace constructors with lowerings that dropped the pin_memory attribute, breaking performance optimizations for CPU-to-GPU data transfers. This regression affected users relying on pinned memory for faster async transfers with CUDA streams.

Authored by jansel and approved by karthickai, the patch restores compiled behavior for both deterministic (torch.tensor) and random (torch.rand) constructors while maintaining Inductor's existing coverage. It builds on prior work from inactive PR #180737 and directly addresses issue #173939. For ML engineers using PyTorch's torch.compile with Inductor, this means pinned memory tensors created inside compiled regions will now correctly allocate host memory that is page-locked (non-pageable), enabling zero-copy transfers to GPUs and reducing latency in data pipelines.

Key Points
  • Fix preserves pin_memory=True for torch.tensor and torch.rand in Inductor-compiled code
  • Resolves issue #173939 where compiled constructors lost pinned memory allocation metadata
  • Follow-up to inactive PR #180737; approved by PyTorch core maintainer karthickai

Why It Matters

Ensures async GPU transfer optimizations work correctly inside PyTorch compiled graphs, critical for high-performance ML pipelines.

📬 Get the top 10 AI stories daily