Boos proposes triple-blind local AI pre-screening to fix peer review crisis

Academic peer review is drowning. NeurIPS 2025 received 21,575 submissions, ICLR 2025 11,603, and ICML 2025 12,107, overwhelming qualified reviewers. In response, many researchers have quietly turned to large language models. An independent analysis of ICLR 2026's 75,800 reviews found roughly 21% fully AI-generated, with over half showing some AI involvement—up from 15.8% in 2024. Key risks include hallucinated citations in accepted papers and hidden prompt-injection instructions embedded in manuscripts that manipulate AI reviewers into giving favorable assessments. Rodrigo Martins Boos's new arXiv paper (2608.14625) proposes a solution: a triple-blind, multi-LLM pre-screening framework developed for a health sciences journal that formalizes and discloses AI involvement while keeping humans in control.

The framework processes each submission through five stages: sanitization/anonymization, parallel AI pre-screening, an automated check gate, blinded human review, and editorial adjudication, with return-to-author loops at the check and editor stages. To address confidentiality concerns behind NIH/NSF bans on feeding unpublished proposals to third-party AI, all three AI reviewers run on locally-hosted, open-weight LLMs within the journal's infrastructure. Boos cites Shen et al., who found five open-source LLMs only achieved 35% exact-match accuracy on quartile classification—supporting the mandatory human adjudication. The system aims to cut the average 13-week wait to first decision while making AI use transparent and regulated. It offers a defensible middle ground in peer review's AI crisis and could scale to other fields.