PyTorch DTensor benchmarks go meta to remove CUDA noise
Meta tensors now benchmark metadata-only ops without CUDA runtime interference.
Deep Dive
The PyTorch team switched to meta tensors for metadata-only DTensor dispatch benchmarks (detach, to/from-local, view, is_same_size) because CUDA runtime host instructions make instruction counts vary across toolkit versions even when dispatch is unchanged. CPU and meta measurements agree within 1%, preserving coverage without device code. Collectives, backward, inplace, and random ops stay on CUDA. This isolates metadata benchmarks from CUDA runtime noise, and the existing 10% baselines already include the new results.
Key Points
- Meta tensors replace CUDA tensors for DTensor benchmarks that only inspect or transform metadata.
- Eliminates up to 10% noise from CUDA runtime host instructions varying across toolkit versions.
- CPU and meta measurements agree within 1%, preserving benchmark coverage without device execution.
Why It Matters
Cleaner, deterministic benchmarks help PyTorch developers optimize DTensor dispatch without device-specific noise.