ScalablePromptus delivers robust video streaming under lossy networks
New dropout training method cuts quality collapse by 82-95% in prompt-based video streaming
Prompt-based video streaming transmits compact semantic prompts instead of pixel data, enabling ultra-low-bitrate communication. However, the existing Promptus framework catastrophically fails under network fluctuation—partially received prompts cause a collapse in video quality. ScalablePromptus, developed by Zehao Cao and colleagues, overcomes this through a dropout training strategy that creates rank-ordered prompt representations. This ensures that even when prompts are arbitrarily truncated, the receiver can reconstruct meaningful video without adaptation. Additionally, the framework incorporates semantic and color-aware prompt inversion for higher fidelity and uses spherical linear interpolation to generate intermediate frames smoothly.
Under stable networks, ScalablePromptus achieves modest quality gains over Promptus. Its true strength emerges in lossy conditions: it reduces the performance degradation caused by truncation by 82%-95% relative to the baseline. This dramatic improvement makes prompt-based streaming viable for real-world deployment where packet loss is common. The method does not require any changes to the receiver side and works with existing generative reconstruction models. By decoupling prompt quality from network reliability, ScalablePromptus paves the way for highly efficient video streaming at bitrates far lower than traditional codecs.
- Dropout training creates rank-ordered prompts that enable graceful quality degradation under packet loss
- Reduces quality collapse by 82%-95% in lossy networks compared to the baseline Promptus
- Combines semantic/color-aware prompt inversion and spherical linear interpolation for high-fidelity reconstruction
Why It Matters
Enables ultra-low-bitrate video streaming robust enough for real-world networks, promising a new era of bandwidth-efficient video communication.