Kunkerkar's queueing model gives GPU clusters formal wait-time guarantees

GPU cluster operators live with unpredictable queue times because existing admission systems rely on greedy heuristics with no formal guarantees. A new paper by Sohan Kunkerkar, posted on arXiv (2607.28223), tackles this head-on by rethinking GPU cluster admission as a multi-class, multi-resource queueing network. The central result is a structural decomposition: pending workloads split into "quotable" ones that have bounded wait times under stability, and "unfeasible" ones that cannot be guaranteed without reconfiguring the cluster. This gives operators a principled way to decide which jobs can be promised an SLA and which cannot.

For quotable workloads, the paper models each cluster queue as an M/G/k system, where the effective server count is derived from a vector packing reduction. The math proves wait times scale as O(1/(1-rho)), where rho is cluster utilization—a clean theoretical result. Kunkerkar also shows that finding the optimal admission order is NP-hard under multi-dimensional resource demands, via reduction from vector bin packing. The theory is validated against Kueue, the standard Kubernetes queueing system, using CPU, memory, and GPU via Dynamic Resource Allocation. The effective k correctly identifies bottleneck resources, Little's Law holds exactly, and the standard Erlang-C formula overestimates waits in the conservative direction. The paper is a significant step toward predictable GPU scheduling, with practical implications for multi-tenant clusters.