Group-of-Latents compresses video to <0.005 bpp with generative AI
Diffusion Transformers deliver stunning perceptual quality at record-breaking compression ratios.
Most video compression algorithms optimize a trade-off between distortion and bitrate, but they struggle at extreme low-bitrates where preserving perceptual quality becomes paramount. Existing methods often introduce artifacts or lose temporal coherence when forced below typical bitrate floors. The new Group-of-Latents (GoL) framework tackles this head-on by shifting from traditional transform-quantization to generative modeling.
GoL operates in the latent space of a causal tokenizer, explicitly separating key intraframe anchors (I-latents) from interframe predictions (P-latents). A Deep Compression Module (I-DCM) encodes I-latents with minimal overhead, while a DiT-based Unified Latent Denoising Module (U-LDM) refines textures and synthesizes P-latents from noise—effectively reconstructing temporal dynamics at zero additional bitrate cost. The result is perceptual fidelity that rivals much higher bitrate codecs, verified on standard benchmarks below 0.005 bpp.
- Operates at extreme bitrates below 0.005 bits per pixel (bpp).
- Uses pre-trained Diffusion Transformer (DiT) priors to generate high-quality spatial and temporal details.
- Synthesizes interframe (P) latents from noise at zero bitrate cost, preserving temporal consistency.
Why It Matters
This could enable high-quality video streaming over extremely low-bandwidth networks, transforming remote monitoring, mobile broadcasting, and edge storage.