Peking Univ and Alibaba's CODA speeds edge video generation 1.8x
New architecture reduces cache bottlenecks, delivering 1.74x energy efficiency on GPUs.
Deploying video diffusion models (VDMs) on edge devices is highly desirable for localized, privacy-preserving content generation, but iterative Transformer-based denoising remains too slow for practical use. Cross-Timestep Caching (CTC) reduces redundant computation by reusing activations across denoising steps, but it creates a massive cache footprint that quickly exceeds on-device VRAM, forcing data into host memory. The interleaved and chain-dependent nature of cache operators with native compute operators leads to repeated PCIe transfers, turning cache reuse into a communication- and serialization-bound bottleneck.
CODA solves this by disaggregating compute and cache paths across the xPU (GPU) and a lightweight DIMM-side near-memory processing engine. It reorganizes fragmented cache activity into coalesced segments and exploits Classifier-Free Guidance (CFG) branch independence to overlap xPU compute with cache-side execution. In experiments, CODA achieved up to 1.80x speedup and 1.74x energy efficiency improvements while maintaining competitive generation quality. The paper has been accepted to MICRO 2026, a top-tier architecture conference.
- CODA disaggregates dense compute and memory-bound cache operators across GPU and a near-memory processor to eliminate PCIe bottlenecks.
- Achieves up to 1.80x end-to-end speedup and 1.74x higher energy efficiency over state-of-the-art caching algorithms.
- Reorganizes cache activity into coalesced segments and overlaps execution using CFG branch independence.
Why It Matters
Enables real-time, privacy-preserving video generation on edge devices without sacrificing performance or energy.