Developer Tools

PyTorch Fixed a Glitch That Made AI Software Slow and Crash-Prone

Behind nearly every AI chatbot is software like this — and it just got more reliable.

Deep Dive

A PyTorch pull request (#197313) stops the ROCm `default` shards from running the same inductor tests twice. The `linux-noble-rocm-py3.11-mi300` default shards were running at roughly 90 percent of their 270-minute step timeout, and timeout failures on `main` rose from about 1 percent through August to 5.6 percent in the week of 2026-09-06 — not because of test growth, since tests per shard had been flat since July. The `default` config picked up every `inductor/*` file, while the `inductor` config re-ran four of them verbatim: about 50 minutes of duplication on a 261-minute shard. The fix came in two commits — making `run_test.py --exclude` match test names exactly, and adding a `tests-to-exclude` input to `_rocm-test.yml`. The PR was authored with assistance from Claude (Anthropic).

Key Points
  • PyTorch was running the same four test files twice on AMD hardware, wasting about 50 minutes out of every 261-minute run.
  • Timeout failures climbed from roughly 1% to 5.6% in a single week — the strain was starting to show.
  • The fix uncovered a test file that had been silently skipped everywhere for years because the skipping tool matched names by prefix, not exact name.

Why It Matters

It won't change your day, but reliable testing keeps AI tools stable and cheaper to run on competing chips.

📬 Get the top 10 AI stories daily