SmellBench: Qwen + Claude score only 50% on code refactoring test

Code agents are getting better at writing functional code, but they often leave behind messy, hard-to-maintain spaghetti. To quantify this, researchers built SmellBench—a benchmark that proactively injects 7 common code smells (like long methods or duplicated code) into clean code from 7 real-world repositories. With 294 controlled refactoring cases across 3 difficulty levels and two instruction settings (explicit vs. implicit), SmellBench evaluates agents not just on functional correctness, but on localization ability and refactoring quality. Human-written ground truths enable precise scoring.

The results are sobering: the best performing combination—Qwen Code as the agent framework with Claude Sonnet 4.5 as the LLM—scored only 50.34 out of 100 on smell elimination. Analysis reveals agents excel at fixing local code smells but struggle with cross-file understanding, meaning they miss global smells that span multiple files. This gap suggests current LLM-based code agents prioritize generating code quickly rather than ensuring long-term maintainability. SmellBench provides a much-needed metric for the community to improve refactoring capabilities.