One Model, Many Skills: Parameter-Efficient Fine-Tuning for Multitask Code Analysis

A team of researchers from the University of Luxembourg and SnT has published a groundbreaking paper titled 'One Model, Many Skills: Parameter-Efficient Fine-Tuning for Multitask Code Analysis.' The study presents the first comprehensive evaluation of multi-task Parameter-Efficient Fine-Tuning (PEFT) for software engineering tasks, demonstrating that a single PEFT module shared across multiple code analysis tasks can match—and sometimes surpass—the performance of full multi-task fine-tuning. This approach updates only a small fraction of a model's weights, achieving accuracy close to single-task fine-tuning while dramatically reducing resource requirements.

The research shows multi-task PEFT can cut computation costs by as much as 85% and reduce storage needs by a factor equal to the number of tasks. The team found that task grouping significantly impacts outcomes, with factors like task stability, model architecture, complementarity, and dataset quality determining success. Notably, their benchmarks reveal that even a 1-billion-parameter model with multi-task PEFT outperforms direct prompting of much larger open-source LLMs like DeepSeek, Qwen, Mistral, CodeLlama, and StarCoder on code analysis tasks, where these general models typically underperform despite their strength in code generation.