SlackServe uses playout slack to prevent stalls in AI video streaming

Researchers from Peking University and collaborators have developed SlackServe, a serving system designed to maintain playout continuity in real-time streaming video generation using autoregressive diffusion transformers (AR-DiTs). Unlike traditional batch inference bound by fixed latency, AR-DiTs generate video chunks sequentially for immediate playout. The critical metric is no longer throughput but “playout slack”—the amount of buffered video ahead of the player. If slack runs out, users see stalls.

SlackServe uses playout slack as a unified signal to drive resource management. It employs three-tier priority queues, re-homing, and elastic sequence parallelism to shift GPU capacity toward streams with dangerously low slack. It also applies a Bi-Modal Pareto Routing mechanism to dynamically select per-chunk fidelity, sacrificing minor quality on some chunks to ensure they are delivered before their deadline. On a 16-H100 GPU cluster, SlackServe improves Continuous Play Ratio (CPR) by 1.64x to 3.29x and reduces Time to First Chunk (TTFC) by 1.61x to 9.65x compared to baselines, while preserving comparable generation quality. This work highlights a fundamental shift in serving objectives for real-time generative AI: timeliness trumps perfection.