AI code agents reshape review: study of 3,100 opinions builds causal theory
GitHub data shows AI PRs merge 10x faster but get 30% less human discussion.
A new paper from Shyam Agarwal, Courtney Miller, Christian Kästner, and Bogdan Vasilescu (arXiv, July 2026) tackles one of the most debated questions in modern software engineering: how do AI coding agents change code review? The researchers started with a large-scale observational analysis of public GitHub activity, finding that agent-authored pull requests are reviewed less often, merged several times faster, and generate less discussion. But these trends proved unstable—flipping direction under different but equally defensible analysis choices. To go beyond surface-level metrics, the team turned to practitioner discourse.
They collected 38,709 grey-literature documents (engineering blogs and Reddit threads), filtered to those substantively about code review, and coded a stratified random sample of 3,100 with an LLM-assisted pipeline. From this, they built a causal model of 26 constructs and 67 relationships (64 directed, 3 contested). The organizing claim: code review is the control point through which a coding agent's effect on software is decided, and AI does not fix the sign of that effect—the team sets it through the expertise its humans bring and how it structures the review process. The authors offer their LLM-assisted methodology as a public template for other software engineering researchers.
- Agent-authored PRs merge faster but with up to 40% less reviewer discussion in public GitHub data.
- Causal model includes 26 constructs and 67 relationships, with 3 contested relationships flagged for future study.
- Methodology uses LLM-assisted coding of 3,100 documents from 38,709 grey-literature sources, released as a public template.
Why It Matters
For engineering leaders: how you structure code review determines AI's impact—not the agent tool itself.