UniVL cuts image generation costs by 52% with unified vision-language embedding

A new paper from researchers Jiayun Wang, Yu Wang, and colleagues presents UniVL, a framework that redefines controllable image generation by fusing vision and language into a single unified embedding. Instead of using separate encoders for a reference image and a text prompt, UniVL takes a single visual input where the textual instruction is rendered onto a spatial mask. The encoder, adapted from an OCR-pretrained backbone, reads this unified condition optically and produces a token sequence that binds semantics to spatial locations. This eliminates the need for a standalone text encoder (like T5) during inference, dramatically reducing computational load.

On the newly constructed UniVL-ImgGen benchmark—477K mask-annotated images—UniVL achieves significant quality improvements: FID drops from 14 to 11 and PSNR rises from 16 to 20. More importantly, it reduces inference TFLOPs by up to 52% and runtime by up to 44% compared to text-prompted baselines. The two-stage pipeline first aligns UniVL with the VAE embedding space, then conditions a pretrained diffusion backbone entirely on UniVL embeddings. This unified paradigm paves the way for more efficient, spatially precise image generation with minimal text interface.