RepoMirage reveals code agents fail 41% on repository context reasoning

A new paper from Tsinghua University researchers introduces RepoMirage, a diagnostic evaluation suite that exposes a critical blind spot in today's code agents. Built on SWE-Bench Verified, RepoMirage applies three types of semantics-preserving repository-level perturbations—renaming variables, restructuring imports, reordering functions—to test whether agents truly reason about code across multiple files or just pattern-match. The results are striking: on standard issue resolution tasks, agents score 66.8%, but when RepoMirage-Extend introduces structural bottlenecks that force cross-file reasoning, accuracy plummets to 25.3%.

The researchers identified 'exploration drift' as the root cause: agents browse more files but fail to synthesize the retrieved information into actionable structure. To fix this, they propose RepoAnchor, a prototype workflow that explicitly separates repository exploration from downstream problem solving. By first building a structural scaffold (dependency graphs, call hierarchies) and then using that scaffold to guide the agent, RepoAnchor recovers much of the lost performance. The findings suggest that current code agents—including those powering GitHub Copilot, Cursor, and Devin—lack robust repository context reasoning, and that structure-aware methods are the next frontier for making AI truly understand large codebases.