Developer Tools

PyTorch's Latest Update Silences a Frustrating vmap Error — and Speeds Up Batch Processing

Fixes frustrating RuntimeError in vmap for torch.repeat_interleave with data-dependent repeats.

Deep Dive

PyTorch now properly handles `vmap` with `torch.repeat_interleave` when the `repeats` argument is a batched tensor. Previously, users got a `RuntimeError: no batching rule implemented for aten::repeat_interleave.Tensor`. The issue, filed as #135424, prevented vmap from working with functions that called `repeat_interleave` with variable-length repeats per batch element. Contributor RoyK0108 added the missing batching rule, co-authored with Claude Opus 4.8 and approved by Skylion007.

The key insight: When `repeats` is batched, the output length is data-dependent (sum of repeats can differ per batch element), so vmap cannot infer a static output shape. The fix requires that users provide the `output_size` argument when calling `repeat_interleave` inside a vmap’d function with batched repeats. The implementation builds gather indices in a fully batched, shape-static manner: e.g., `marks[b, cumsum(repeats[b])] += 1` followed by a cumulative sum. If `output_size` is omitted for batched `repeats`, a clear error is raised. The PR includes comprehensive tests covering indices overload, self_Tensor overload, unbatched-repeats fallback, and the missing-output_size error case.

Key Points
  • Fixes RuntimeError when vmap-ing functions that use torch.repeat_interleave with batched repeats tensor
  • Requires user to provide output_size parameter when repeats is batched to enable static shape inference
  • Uses a novel gather indices algorithm with cumsum, verified against reference for edge cases like leading/trailing zeros

Why It Matters

Enables vmap to handle data-dependent tensor operations, unlocking more flexible functional transforms in PyTorch.

📬 Get the top 10 AI stories daily