ICML 2026 paper: Data selection bias can accelerate model collapse on synthetic data

A paper accepted at ICML 2026, 'When Sample Selection Bias Precipitates Model Collapse' by Xinbao Qiao et al., investigates how data selection practices can inadvertently worsen the widely reported phenomenon of model collapse — where recursive training on synthetic data erodes distributional tails and homogenizes outputs. The authors identify a critical but overlooked scenario: when verifiers used for data selection operate in low-resource environments (e.g., separate healthcare data silos or proprietary financial databases) and have access only to small, fragmented, and biased slices of the target manifold. In such settings, selection preferentially retains samples aligned with the local manifold while pruning globally relevant tail modes, transforming a supposed safeguard into a collapse accelerator.

The paper provides theoretical proof that such siloed selection accelerates collapse and induces a power-law decay in diversity. As an initial mitigation, the authors propose constructing Wasserstein proxy references from multiple silos without requiring raw data to be shared. Empirical results confirm that local-reference selection fails on skewed distributions, whereas collaborative proxy references significantly mitigate diversity degradation. The work serves as a caution for any AI pipeline relying on recursive synthetic data training, especially in domains where real data coverage is fragmented or scarce. The paper will be presented at ICML 2026.