New research exposes flaws in AI generative model training
Factorized sampling in diffusion models fails 84% of the time despite perfect-looking stats
A new paper from Duong Bach, Hai Nguyen Hong, and Cuong Do titled 'Marginal Matching Does Not License Factorized Sampling' reveals critical flaws in how AI generative models handle conditional independence. The researchers demonstrate that common training practices - which match latent style variables to Gaussian priors - fail to ensure true independence between style and class information.
Their audit of four baseline models and a case-study model showed that despite achieving near-zero global Maximum Mean Discrepancy (MMD) scores (indicating perfect marginal distribution matching), linear probes could still recover class labels with 74-100% accuracy (vs. 10% chance level). Even more concerning, externally evaluated class-conditional generation succeeded only 16% of the time on CIFAR-10, despite the models achieving 99.15% clustering accuracy.
- Factorized generative models can appear statistically perfect (near-zero MMD) while still leaking class information into 'style' variables
- Linear probes recovered class labels with 74-100% accuracy despite perfect-looking marginal distributions in four baseline models
- Class-conditional generation succeeded only 16% of the time on CIFAR-10, even when models achieved 99.15% clustering accuracy
Why It Matters
This research exposes fundamental flaws in how we validate conditional independence in generative AI models, potentially undermining trust in their outputs.