New arXiv study: 99% of AI agent skill files contain 'skill smells'

Agent Skills are on-demand domain knowledge modules for LLM agents, defined by a mandatory SKILL.md file containing metadata and unstructured Markdown. Despite rapid adoption, no systematic study has examined how these files are authored or whether best practices exist. In a new empirical study, researchers from the University of California, Irvine, analyzed 238 real-world SKILL.md files from public repositories. They qualitatively coded the content and derived a taxonomy of 13 high-level semantic components (e.g., goal, input, output, examples) and 44 lower-level components. Via a multivocal literature review of 29 sources, they identified authoring best practices and defined 'skill smells' as violations of those practices.

The team then built an automated smell detector and applied it to the dataset. Results show that over 99% of SKILL.md files contain at least one skill smell, with many containing multiple. Common smells include missing required metadata fields, overly verbose or vague descriptions, and lack of structured examples. Once introduced, skill smells rarely disappear as skills evolve — the study found no significant improvement across skill versions. This gap between recommended and actual authoring practices highlights a critical quality issue for the growing ecosystem of AI agent skills. The paper motivates the need for automated remediation tools and developer education to improve skill maintainability and reliability.