CodeScaler: Scaling Code LLM Training and Test-Time Inference via Execution-Free Reward Models

A research team led by Xiao Zhu has introduced CodeScaler, a novel method for scaling the training and inference of code-generating large language models (LLMs) without relying on executing unit tests. The core innovation is an 'execution-free reward model' that uses carefully curated preference data and syntax-aware techniques to judge code quality, bypassing the traditional bottleneck of needing reliable, high-quality test cases for reinforcement learning.

Technically, CodeScaler incorporates syntax-aware code extraction and validity-preserving reward shaping to ensure stable optimization. In benchmark tests, it boosted the performance of the Qwen3-8B-Base model by an average of +11.72 points, surpassing binary execution-based reinforcement learning by +1.82 points. Crucially, it achieved this while providing a 10-fold reduction in inference latency compared to unit test approaches, making real-time code assistance significantly faster.

The context for this breakthrough is the current reliance on Reinforcement Learning from Verifiable Rewards (RLVR), which depends on execution feedback from tests. CodeScaler's execution-free approach not only matches this performance but extends capabilities to synthetic datasets completely lacking test cases. Furthermore, it demonstrated surprising generalizability, outperforming existing reward models on the RM-Bench by +3.3 points in the code domain and +2.7 points on average in general and reasoning domains.

For developers and AI practitioners, this means faster, more scalable training of code LLMs without the overhead of creating exhaustive test suites. It opens the door to training on larger, more diverse code corpora and enables lower-latency coding assistants that can provide high-quality suggestions without the computational cost of code execution.