AI Code Self-Review Triggers Model Collapse, New Study Shows

A new paper from researchers Xinyuan Song, Zekun Cai, and Liang Zhao (arXiv:2606.28438) examines the risks of recursive self-training in code LLMs. As AI-generated code enters real repositories and later becomes training data, a loop forms that can degrade model quality. The study compares three regimes: no review, human-gate review using filters like compilation and static quality checks, and AI-self-gate review using the model’s own signals (perplexity and binary self-scoring). Across multiple code LLMs and benchmarks, no review collapses fastest. Human-gate filters slow collapse but cannot stop it entirely. AI-self-gate filters look promising early but later lose their filtering effect, entering a 'rubber-stamp' regime where acceptance scores rise while actual benchmark correctness falls.

The authors explain this phenomenon by framing review as gated distributional reweighting. They prove that AI self-gating degenerates to ungated self-training under a self-confirming acceptance condition. Their spectral analysis of representation-level covariance concentration shows how recursive retraining concentrates model representations, reducing diversity and making errors self-reinforcing. The paper concludes that stable recursive code LLM training requires exogenous verification—human oversight or independent filters—rather than model-coupled self-review. As AI coding tools accelerate production, this finding has critical implications for software engineering pipelines where AI reviews AI-generated code without external checks.