Open LLM Code Llama fine-tuned for automated code reviews

Smitha S Kumar and her team have introduced a fine-tuned model aimed at enhancing automated code review feedback using the open LLM Code Llama. Their research highlights the effectiveness of parameter-efficient fine-tuning (PEFT), which leverages insights from a more capable model to improve feedback on buggy Java code. Their findings indicate that PEFT not only enhances the quality of feedback but also significantly outperforms traditional prompt engineering methods. The results were validated through student evaluations, manual annotations, and automated metrics such as BLEU and ROUGE.

The study's implications are substantial for programming education, suggesting that fine-tuned models can provide valuable, scalable feedback tools that help guide student learning. Participants reported that the PEFT model's feedback was as effective as that from proprietary models like ChatGPT, with suggestions for further enhancements, such as clearer explanations of technical terms. This research opens avenues for developing free, deployable feedback systems that support critical thinking and improve educational outcomes in software engineering.