SkillJuror reveals skill organization boosts LLM agent success by 4.1%
Progressive Disclosure triples skill resources touched per agent trajectory in 82-task study.
A new paper from Zhiyu Chen and six co-authors presents SkillJuror, a framework for measuring how the organization of agent skills changes runtime behavior in large language model agents. The study distinguishes between what a skill says and how it is structured, introducing Progressive Disclosure—a paradigm where a concise root file directs agents to supporting resources on demand—and comparing it to a normalized flat baseline. Using SkillJuror, the researchers ran 410 matched multi-trial evaluations across 82 tasks from a new benchmark called SkillsBench.
Results show that Progressive Disclosure significantly alters agent behavior before aggregate outcomes: distinct skill resources touched per trajectory rose from 1.18 to 3.85, and effective uptake events from 1.33 to 3.92. This led to 17 more verifier-passing trials (+4.1%) over the flat baseline. However, the benefit was task-dependent—Progressive Disclosure helped when resources guided implementation, checking, or repair, but was weaker when success hinged on exact output conventions, numerical thresholds, or long artifact-generation pipelines. The findings underscore that skill organization is not mere presentation; it changes how agents search and apply procedural knowledge, with outcome gains tied to how actionable the exposed resources are for the task.
- SkillJuror framework measures how skill organization (e.g., Progressive Disclosure vs flat) changes LLM agent runtime behavior.
- Progressive Disclosure increased distinct skill resources touched per trajectory from 1.18 to 3.85 and effective uptake from 1.33 to 3.92.
- 17 additional verifier-passing trials out of 410 (+4.1%) over flat baseline, but gains are task-dependent.
Why It Matters
SkillJuror shows that how you organize agent skills can boost performance by 4.1%, guiding future agent design.