Chen's code-first peer review uses AI to audit papers, not narratives
AI shifts peer review from polished stories to executable code evidence.
In a new arXiv paper (arXiv:2606.07683), Jienan Chen introduces a radical vision for peer review in computational sciences: code-first review. The current system relies on author-written manuscripts that frame narratives, often hiding implementation flaws or unsupported claims. Chen's protocol requires authors to submit executable code, data, configurations, and a minimal claim manifest. A venue-controlled AI system then automatically builds the environment, executes experiments, audits code paths, maps claims to concrete evidence, and produces a standardized Review Package for human reviewers.
The system formalizes a claim-evidence contract and defines abstractions like the Generated Review View. Chen argues this shifts focus from storytelling to reproducible evidence, without replacing human judgment. The paper also addresses governance challenges: AI bias, prompt injection, model instability, auditability, and author appeal. At 17 pages, it's a detailed blueprint for transforming how computational research is validated, potentially making peer review more rigorous and transparent.
- Authors submit executable code, data, and claim manifests instead of narrative manuscripts.
- AI system builds environment, runs experiments, audits code paths, and maps claims to evidence.
- Paper addresses AI bias, prompt injection, and model instability as key governance challenges.
Why It Matters
Could make computational peer review rigorous, reproducible, and less vulnerable to narrative spin.