SkillComposer automates reusable skills for natural-language robot programming
LLM-driven robot programming that reuses successful code—12 users saw higher success rates.
SkillComposer, developed by John Woods and Hasti Seifi, tackles a key bottleneck in natural-language robot programming: LLMs handle simple commands but often fail on multi-step tasks, can't decompose high-level instructions, and rarely reuse prior solutions. The system introduces a generate-test architecture in which an LLM iteratively generates and revises robot programs before execution in simulation. Once a program runs successfully, an online library-learning algorithm compresses recurring function sequences into macro skills, enabling the system to reuse those abstractions for future tasks.
The approach was evaluated through ablation experiments and a user study with 12 participants performing manipulation and robot caregiving tasks. Results showed that evaluator-guided generation and learned abstractions significantly improved success rates and usability, while cutting user effort in natural-language programming. SkillComposer is being submitted to IEEE Humanoids 2026, and the paper is available on arXiv with code and media links. For robotics developers, this points toward LLM-driven systems that continuously learn and reuse skills, making complex robot programming more accessible to non-experts.
- SkillComposer uses a generate-test loop where an LLM iteratively revises robot programs until they run successfully in simulation.
- Successful programs are compressed into reusable macro skills via an online library-learning algorithm, enabling future task reuse.
- In a 12-participant user study on manipulation and caregiving tasks, evaluator-guided generation improved success rates and usability while reducing user effort.
Why It Matters
Makes complex robot programming more accessible by letting LLMs learn reusable skills, cutting user effort for multi-step tasks.