Research & Papers

Dream team's SALSA detects AI-generated code with 2.6x better F1 score

New single-pass LLM method surpasses CodeBERT by 158% on out-of-distribution code.

Deep Dive

Large language models are increasingly generating code, raising concerns about authorship, academic integrity, and software supply chain trust. To address this, researchers from the Dream team submitted SALSA (Single-pass Autoregressive LLM Structured Classification) to SemEval-2026 Task 13 Subtask A, which requires binary classification of code snippets as human-written or machine-generated, with a focus on out-of-distribution (OOD) generalization across unseen languages and domains.

SALSA avoids hand-crafted features by delegating the authorship decision entirely to the LLM. It maps each class to a dedicated output token and trains the model to emit a single-token label in a structured response. The team combined balanced sampling across programming languages with parameter-efficient fine-tuning and conservative training (low learning rate, single epoch) to prevent overfitting. This approach achieved an OOD F1 score of 0.789 on the official leaderboard, substantially outperforming the CodeBERT baseline (F1=0.305) by a factor of 2.6.

The results demonstrate that a carefully fine-tuned autoregressive LLM can reliably detect machine-generated code across unseen languages without requiring engineered features or complex decision rules. This is particularly important as AI-assisted coding tools become ubiquitous, and institutions need automated ways to verify code provenance. The Dream team's SALSA method provides a practical, scalable solution that leverages the model's own understanding of code style and structure.

Key Points
  • Achieves OOD F1 score of 0.789, far surpassing CodeBERT's 0.305 baseline.
  • Uses single-token label emission and balanced sampling across programming languages.
  • Avoids hand-crafted features by delegating authorship decision to the fine-tuned LLM.

Why It Matters

Enables reliable detection of AI-generated code, critical for academic integrity and software supply chain security.

📬 Get the top 10 AI stories daily