RLPF trains Qwen3-32B to write 5x more efficient code

Code generation models are typically trained only to produce correct outputs, leaving massive performance gaps overlooked. RLPF turns execution outcomes into a staged reward: failed programs are ordered by execution progress, while correct programs are ranked by their relative improvement from a baseline toward an expert reference. This design provides useful learning signals even before correctness is achieved, and then shifts to runtime-optimization pressure after programs pass.

Fine-tuning Qwen3-32B with RLPF on PerfCodeBench dramatically improved results, boosting correct-and-runnable solutions from 11.1% to 54.6% and relative efficiency from 8.1% to 38.6%, making the model competitive with stronger open-weight systems. The optimization behavior transferred modestly to EffiBench-X, and experiments showed model-generated references offer useful but weaker supervision than expert references. The composite reward proved more reliable than correctness-only or runtime-only baselines, suggesting code agents can be trained not just to pass tests, but to write genuinely faster programs.