Novice Developers Produce Larger Review Overhead for Project Maintainers while Vibe Coding
Analysis of 22,953 GitHub PRs shows low-experience 'vibe coders' shift verification burden onto teams.
A new study from researchers at the University of Calgary, accepted to the MSR 2026 Mining Challenge, provides crucial data on the real-world impact of AI-assisted coding. Analyzing 22,953 Pull Requests (PRs) from 1,719 developers in the AIDev dataset, the research tackles a pressing question for project managers: can less-experienced developers using AI coding agents (a practice dubbed 'vibe coding') effectively substitute for expert developers? The findings suggest a significant trade-off between speed of code generation and the burden of code review.
The study split developers into lower-experience (Exp_Low) and higher-experience (Exp_High) groups. The data reveals that Exp_Low developers, while generating code quickly with AI agents, produce PRs that are 2.15x larger in commits and change 1.47x more files. This volume translates directly into a 4.52x increase in review comments from maintainers, a 31% lower PR acceptance rate, and PRs that remain unresolved 5.16x longer. The conclusion is that low-experience vibe coders shift the verification burden onto reviewers, creating a substantial 'review overhead.' For practice, the authors warn that replacing experienced developers with novices using AI requires increasing review capacity and suggest combining targeted training with adaptive PR review cycles to manage the new workflow dynamics.
- Low-experience AI coders submit PRs 2.15x larger in commits and changing 1.47x more files than experienced peers.
- Their PRs generate 4.52x more review comments and have a 31% lower acceptance rate, staying open 5.16x longer.
- The study analyzed 22,953 PRs from 1,719 developers, concluding AI shifts verification burden to reviewers, increasing team overhead.
Why It Matters
Managers cannot simply replace senior devs with AI-assisted juniors without planning for a major increase in code review resources.