PyTorch caps clang-tidy parallelism to prevent CI OOM crashes
PyTorch's CI was crashing because clang-tidy spawned too many processes per host cores.
PyTorch's CI team resolved a persistent out-of-memory (OOM) issue affecting clang-tidy and clang-format linting jobs on Kubernetes ARC runners. The root cause: the clang adapters used os.cpu_count() to determine parallelism, but on k8s pods this returns the host node's physical core count — often significantly higher than the pod's allocated CPU limit. For an "--all-files" run, this resulted in far more memory-hungry clang-tidy processes being launched than the pod could handle, leading to OOM kills and broken CI.
To fix it, they introduced a -j/--num-workers flag for the adapters that defaults to the MAX_JOBS environment variable (with os.cpu_count() as fallback). In the clang lint job, they set MAX_JOBS="$(nproc --ignore=2)" — exactly as _linux-build.yml does — so nproc correctly reports the pod's cgroup-constrained CPU budget. This caps the number of concurrent clang-tidy processes to a safe limit, preventing memory exhaustion. The patch (PR #187535, approved by @atalman and @huydhn) was authored with assistance from Claude Code.
- clang-tidy and clang-format spawn one process per os.cpu_count(), but on k8s that returns the host node's core count, not the pod's limit, causing OOM.
- Added a -j/--num-workers flag defaulting to the MAX_JOBS env var; set MAX_JOBS=$(nproc --ignore=2) in the clang lint job to cap parallelism to the pod's real CPU budget.
- PR #187535 was authored with assistance from Claude Code and approved by two PyTorch maintainers.
Why It Matters
Stops CI crashes from memory overload, saving PyTorch developers time and ensuring stable automated linting.