New study: 95% of Gemini CLI runs exploited via malicious skill files

A new arXiv preprint, "Towards a Risk Assessment of Malicious Skill Files in Coding Agents," reveals a severe security vulnerability in autonomous coding agents. The researchers—Rui Yang, Michael Fu, Kla Tantithamthavorn, Chetan Arora, and Joey Chua—focused on the "skills interface": folders of instructions and scripts that agents dynamically load to specialize behavior. They demonstrated that this interface can be weaponized by embedding malicious shell commands inside natural-language skill files, which appear benign to both the agent and human reviewers.

To prove the threat, the team developed an adversarial skill-synthesis method using six LLMs from four families to transform 471 real-world shell commands into 2,826 malicious skills, mapped to 11 MITRE ATT&CK tactics. This dataset is released as a benchmark. Their evaluation pipeline used stratified runs, evidence anchoring, a refusal veto, and a deterministic declared-intent override, validated against a blind human gold standard with high agreement (Cohen's kappa = 0.85). Testing two enterprise-grade agents—Gemini CLI and Qwen Code—over 5,629 completed runs, they found exploitation rates of 95.5-96.1% for Gemini CLI and 71.6-74.0% for Qwen Code. Explicit safety recognition occurred in only 1.99% of runs, meaning agents rarely flagged the malicious intent. The findings suggest that current agents lack adequate defenses against this attack vector. The authors urge enterprises to assess and mitigate skill-interface risk before deploying coding agents in production environments. Code and dataset are publicly available via the paper's link.