PyTorch makes topk deterministic with stable tie-breaking
CPU, CUDA, and ROCm now guarantee reproducible top-k results
PyTorch's latest commit makes the topk operation fully deterministic under the deterministic algorithms flag. Previously, topk could produce non-reproducible results when tied values existed, because the selection order depended on parallel execution order. The new behavior ensures that when `torch.use_deterministic_algorithms(True)` is set, tied values are resolved with stable index tie-breaking. This is critical for debugging, scientific reproducibility, and compliance in regulated industries.
Implementation details vary by backend. On CPU, the algorithm now compares by value first, then by index, guaranteeing a fixed order for ties. On CUDA, instead of a full fallback sort, PyTorch keeps the existing top-k selection path and applies stable sorting only to the selected top-k outputs, avoiding the overhead of sorting the entire tensor on NVIDIA hardware. For ROCm, the current top-k gather reserves tie slots through atomics, so a stable sort is used in deterministic mode. The result: fast default kernels remain available, while deterministic mode activates stable algorithms only when requested.
- CPU topk now compares by value then index in deterministic mode for stable tie-breaking
- CUDA avoids full sort fallback; stable sorting is applied only to the selected top-k outputs
- ROCm uses a stable sort in deterministic mode due to atomic-based tie slot reservations
Why It Matters
Ensures reproducible AI experiments across hardware, critical for debugging and compliance.