SkelDPO boosts code efficiency by 2-7% via skeleton-guided optimization
New training method improves code generation efficiency without sacrificing correctness.
Researchers Yu Yu and Chen Lyu have introduced SkelDPO (Skeleton-Guided Direct Preference Optimization), a novel framework that enhances the efficiency of code generated by Code Large Language Models. Traditional methods treat entire programs as single optimization targets, missing structural factors that influence runtime performance. SkelDPO addresses this by first identifying efficient and inefficient implementations from code datasets, then locating efficiency-prone and inefficiency-prone points through comparative analysis. During training, it uses a joint code and skeleton preference loss, allowing the model to learn both semantic correctness and efficiency-critical components. Results show consistent improvements over existing SOTA methods: Pass@1 improves by 3-6%, Beyond@1 by 3-7%, and Effi@1 by 2-5%, with larger gains on complex tasks.
This approach breaks the limitation of conventional preference optimization that relies solely on correctness or efficiency pairs, offering a new perspective on skeleton-level efficiency alignment. The framework is publicly available, including all datasets and source code (arXiv:2606.06826).
- SkelDPO improves Pass@1 by 3-6% and Effi@1 by 2-5% over SOTA methods
- Uses skeleton-guided preference loss to model structural efficiency factors
- Publicly available datasets and code on arXiv (2606.06826)
Why It Matters
Makes code LLMs faster and cheaper to run, critical for production deployments.