Developer Tools

PyTorch fixes hardtanh export bug with new Meta kernels

torch.export now handles inverted bounds for hardtanh without breaking legacy behavior.

Deep Dive

PyTorch's export pipeline encountered a subtle divergence between eager and exported models when using the hardtanh activation function. The `torch.export` path runs `aten.hardtanh` through FakeTensor/Meta dispatch to trace the graph. Without a dedicated Meta kernel, the system fell back to the `_refs.nn.functional.hardtanh` decomposition, which intentionally follows the Python frontend's validation and rejects inverted bounds where `min_val > max_val`. However, the native `aten.hardtanh` operator has legacy behavior that accepts such inverted bounds and delegates the result to clamp. This caused export to fail for models that eager ATen accepted, creating a bug for users relying on that legacy behavior.

The fix, implemented in PR #185298, adds dedicated Meta kernels for `aten.hardtanh`, `aten.hardtanh.out`, and `aten.hardtanh_`. These kernels mirror the native ATen validation order for bool/complex inputs, integer scalar conversion, unsigned negative limits, out device/dtype errors, scalar range checks, and resized out strides. By scoping the fix entirely to the Meta kernel layer, the PR preserves eager behavior and the Python frontend without BC-breaking changes. The alternative of modifying the native hardtanh or the ref frontend was rejected to avoid blurring the distinction between `torch.nn.functional.hardtanh` and `torch.ops.aten.hardtanh`. The fix is tested via `test/test_meta.py -k hardtanh` and `test/export/test_export.py -k test_export_allows_aten_hardtanh_with_inverted_bounds`, resolving issue #161081.

Key Points
  • Bug: torch.export failed when aten.hardtanh had inverted bounds (min_val > max_val) due to decomposition fallback.
  • Fix: Three new Meta kernels (hardtanh, hardtanh.out, hardtanh_) preserve native ATen semantics during export.
  • Design: Scoped to export path only, avoiding BC-breaking changes to eager mode or Python frontend behavior.

Why It Matters

Ensures backward compatibility and accurate export for PyTorch models using hardtanh with non-standard bounds.

📬 Get the top 10 AI stories daily