WildShadowRemover uses video diffusion models to erase shadows in real-world footage

A team of researchers has introduced WildShadowRemover, a novel framework for removing shadows from in-the-wild videos using a pretrained video diffusion model adapted via LoRA fine-tuning. The approach addresses the longstanding challenge of video shadow removal under complex, unconstrained real-world illumination. To maintain high-frequency textures while leveraging the generative prior of diffusion models, the authors augment the frozen VAE decoder with a detail injection module and a shadow-mask-guided frequency-decomposed modulation module. This selectively restores fine details and suppresses shadow artifacts. Additional geometry-aware guidance comes from monocular depth priors using Depth Anything 3, helping the model handle challenging lighting conditions.

Alongside the model, the team constructed WildShadow, a large-scale paired video shadow removal dataset covering diverse synthetic scenes, providing a benchmark for future research. Extensive experiments show that WildShadowRemover outperforms existing methods in both shadow removal quality and temporal consistency, producing temporally coherent shadow-free videos with superior visual quality. The framework generalizes well across a variety of challenging in-the-wild scenarios, making it a significant step forward for video editing, augmented reality, and other vision applications that require clean, shadow-free footage.