LEACL uses LLMs to automate curriculum learning for robotic manipulation

Long-horizon manipulation tasks in robotics are notoriously difficult for reinforcement learning due to sparse rewards and prolonged action sequences. Traditional automatic curriculum learning (ACL) tackles this by training agents from easy to hard tasks, but success depends on manually crafted task parameter spaces and difficulty measures that don't generalize well. Recent work with LLMs offers a way to break complex tasks into meaningful subtasks using common-sense knowledge, yet these methods still rely on hand-designed dense reward functions for each subtask, introducing bias and requiring significant human effort.

LEACL combines the strengths of LLMs and ACL to overcome these limitations. First, an LLM decomposes a long-horizon task into a sequence of subtasks and automatically generates task-dependent specifications—such as initial states and goal parameters—for each subtask. These specifications feed into an ACL algorithm that trains the agent using only sparse rewards (e.g., success/failure signals), completely eliminating the need for dense reward engineering. Evaluated on five tasks from the LIBERO benchmark, LEACL matches or surpasses the asymptotic success rates of systems using human-designed dense rewards, while requiring far less human input. This work suggests that LLM-driven curriculum generation can make robot learning more scalable and generalizable.