Researchers unveil SkillTrace for auditing AI skill reuse
New framework detects LLM-agent skill theft with 93.8% accuracy
Researchers from Zhejiang University and Tsinghua University unveiled **SkillTrace**, a multi-trace provenance auditing framework designed to detect skill reuse in LLM-agent ecosystems. Unlike traditional code clone detection, SkillTrace tracks three distinct provenance traces—Expression, Implementation, and Operational—captured through a Skill Operational Graph (SOG) that models activation flows, procedures, and resource dependencies. The Operational Trace extraction is LLM-assisted only during ingestion, while audits rely on deterministic trace comparisons with calibrated negatives to identify reuse patterns.
On the **SKILLTRACE-BENCH** dataset (820 transformed reuse positives, 100 marketplace anchors, 751 negatives), SkillTrace achieved **0.938 AUROC** and **0.898 F1**, outperforming single-modality baselines. A wild audit of **36,446 marketplace skills** revealed that trace-attributed evidence uncovered actionable reuse review queues that repository-level tools missed. The framework addresses a critical gap as LLM-agent skills—bundles of metadata, instructions, code, and workflows—become marketplace artifacts, where partial reuse often evades traditional detection methods.
- SkillTrace audits LLM-agent skills across 3 traces: Expression, Implementation, and Operational (via a Skill Operational Graph)
- Achieves 0.938 AUROC and 0.898 F1 on a benchmark of 820 positives vs. 751 negatives
- Identifies actionable reuse queues in 36,446 real-world skills beyond repository-level baselines
Why It Matters
This framework provides the first rigorous way to audit LLM-agent skill reuse in marketplaces, preventing IP theft and ensuring fair attribution for AI workflows.