PerfAgent uses profiler feedback to double code optimization success rates
LLM agents now find real bottlenecks and achieve expert-level speedups reliably.
PerfAgent, developed by Ryan Deng and colleagues, tackles a critical gap in LLM-based code optimization: while agents excel at correctness tasks (like fixing bugs or implementing features), they struggle with repository-level performance optimization that preserves behavior. The system integrates a profiler that identifies real hotspots hidden behind abstraction layers and native extensions, then feeds this evidence back to a verifier-in-the-loop workflow that decides what to optimize next—not just timing but actual profiler data. This iterative refinement continues beyond the first passing patch, ensuring deep speedups without breaking edge cases.
On the GSO and SWE-fficiency-Lite benchmarks, PerfAgent more than doubled the rate of expert-matching patches compared to OpenHands with GPT-5.1: from 19.6% to 39.2% on GSO and from 26% to 74% on SWE-fficiency-Lite. Remarkably, it also outperformed an oracle best-of-five baseline at substantially lower cost, proving the gains come from smarter feedback rather than brute-force sampling. This suggests that profiler-guided iterative refinement is a scalable approach for real-world code optimization tasks where both correctness and performance matter.
- PerfAgent combines profiler feedback with verifier-in-the-loop to find real code bottlenecks and avoid shallow speedups.
- On GSO benchmark, expert-matching patches rose from 19.6% to 39.2%; on SWE-fficiency-Lite, from 26% to 74%, against OpenHands with GPT-5.1.
- Surpasses oracle best-of-five baseline at lower cost by using profiler evidence instead of additional test-time sampling.
Why It Matters
Automated code optimization at repository scale can significantly reduce developer effort and improve application performance.