LatentBurst: A Fast and Efficient Multi Frame Super-Resolution for Hexadeca-Bayer Pattern CIS images
New AI turns blurry phone photos into sharp, high-res images in real-time.
Researchers Sangwook Baek, Vin Van Duong, Karam Park, and Pilkyu Park have introduced LatentBurst, a novel multi-frame super-resolution (MFSR) network designed specifically for hexadeca-Bayer pattern contact image sensors (CIS). This architecture tackles the unique challenges of these advanced sensors—where color filter patterns are denser and more complex than traditional Bayer arrays—by integrating demosaicing, denoising, multi-frame fusion, and super-resolution into a single efficient pipeline. The key innovation lies in its pyramid alignment and fusion approach in latent feature space, which corrects for large object motion and camera shake that typically cause blur or ghosting in burst photography.
LatentBurst is built on an efficient UNet-based structure optimized for real-time mobile inference, making it practical for smartphone cameras. To further boost quality, the team fine-tuned optical flow estimation and applied two-step knowledge distillation, reducing domain gaps between synthetic training data and real-world images. In experiments across various scenarios, LatentBurst outperformed state-of-the-art methods in both resolution and artifact reduction. This breakthrough promises sharper, clearer photos from mobile devices without the computational overhead of traditional multi-frame techniques.
- LatentBurst uses pyramid alignment in latent space to handle large motion and reduce ghosting artifacts.
- The UNet-based network runs efficiently in real-time on mobile devices, integrating demosaicing, denoising, fusion, and super-resolution.
- Two-step knowledge distillation fine-tunes optical flow estimation, bridging domain gaps from synthetic to real images.
Why It Matters
LatentBurst brings pro-level super-resolution to smartphones, enabling crisp burst shots without bulky hardware.