Reddit user finds flow matching beats diffusion for training from scratch

A Reddit user known as TensorForger conducted a hands-on comparison of diffusion and flow matching generative models, training both from scratch under identical conditions to isolate the impact of the training objective. The architecture was a scaled-down UNet with attention blocks, inspired by SDXL, using CLIP ViT-L as the text encoder and FLUX.2’s VAE. The dataset was COCO-2017, containing about 500K image-text pairs. Each model was trained for approximately 12 hours on a single RTX 5090 until convergence. The goal was to sidestep confounds like differing architectures or datasets that plague cross-paper comparisons.

Flow matching dramatically outperformed diffusion during early training: it began generating recognizable shapes (e.g., dog-like forms on grass) before epoch 1, while diffusion produced a green blurry mess for roughly 3 epochs. Flow matching also showed better global structure and prompt guidance, with classifier-free guidance (CFG) having a larger positive effect. Even with much lower CFG scales, flow models achieved better stability and adherence than diffusion models. Perhaps most striking: flow matching exhibited significantly stronger zero-shot generation of unseen combinations—despite using the same text encoder as the diffusion model. The user notes this is anecdotal but invites community testing by offering to upload the weights.