Measuring and Exploiting Confirmation Bias in LLM-Assisted Security Code Review

A new research paper from Dimitris Mitropoulos, Nikolaos Alexopoulos, Georgios Alexopoulos, and Diomidis Spinellis exposes a critical vulnerability in AI-powered security tools. The study, 'Measuring and Exploiting Confirmation Bias in LLM-Assisted Security Code Review', demonstrates that Large Language Models (LLMs) like those powering GitHub Copilot and Claude Code are highly susceptible to confirmation bias. In controlled experiments on 250 known vulnerability and patch pairs, simply framing a code change as 'bug-free' or a 'security improvement' in the prompt caused vulnerability detection rates to plummet by 16% to 93%. The effect was most severe for injection flaws, while memory corruption bugs were slightly more resilient.

The researchers then conducted a practical exploit, mimicking an adversarial pull request that reintroduced a known vulnerability while being framed as an urgent fix. This attack succeeded 35% of the time against GitHub Copilot in a one-shot scenario. Alarmingly, against an autonomous agent like Claude Code configured in a real project pipeline, where an adversary could iteratively refine their misleading framing, the success rate soared to 88%. The study also identified effective countermeasures: redacting potentially biased metadata from prompts and using explicit instructions to consider security risks restored detection in all interactive cases and 94% of autonomous cases. This highlights that the deployment model—interactive assistant vs. autonomous agent—significantly impacts risk.