trunk/1c42537a4ab20aa6abf147380947b366645ab212: [retrybot] Add Initialize containers to retryable step names (#180082)
A single-line code change now ensures infrastructure failures are automatically retried, saving developer time.
A subtle but impactful bug in PyTorch's continuous integration system has been fixed by contributor Huy Do. The issue resided in the project's automated 'retrybot,' a system designed to automatically re-run failed CI/CD jobs that are likely due to temporary infrastructure problems, not code errors. The bug caused the bot to incorrectly skip retries for jobs that failed during the 'Initialize containers' step—a clear infrastructure issue—if a subsequent step like 'Print remaining test logs' also failed. This cascading failure triggered a heuristic designed to catch user-caused test failures, mistakenly classifying the entire job as a non-retryable error.
The fix, merged into the PyTorch main branch (trunk/1c42537a4ab2), is elegantly simple: it adds the string 'Initialize containers' to the bot's internal list of `retryable_step_names`. This change ensures that any failure occurring at that specific initialization phase will always trigger an automatic retry, regardless of what happens in later steps. For the massive PyTorch project with thousands of daily CI runs, this prevents legitimate infrastructure glitches—like network timeouts or container orchestration hiccups—from being flagged as developer errors, saving significant manual review time and compute cycles.
- Bug caused PyTorch's retrybot to skip retries for 'Initialize containers' infrastructure failures.
- Fix adds one step name to a configuration list, ensuring automatic retries for genuine infra issues.
- Prevents wasted developer time and compute resources by correctly classifying failure types in CI/CD.
Why It Matters
For large-scale projects, robust CI/CD automation is critical; this fix reduces noise and improves development velocity.