Active Reward Machine Inference From Raw State Trajectories

A team of researchers from the University of Michigan and the University of Oxford has published a paper titled 'Active Reward Machine Inference From Raw State Trajectories' on arXiv. The work tackles a core bottleneck in robot programming: manually specifying reward machines. These are automaton-like structures that define the sub-tasks and memory needed for a robot to complete a complex, multi-stage job. The new method learns the complete reward machine structure directly from the raw sequences of states a robot experiences and the actions it takes, operating in what the authors call an 'information-scarce regime' with no access to the actual rewards, pre-defined high-level labels, or the internal nodes of the machine.

The technique is further enhanced with an active learning component, where the system can strategically query for extensions to existing trajectories. This improves both data and computational efficiency by focusing learning on the most informative parts of the task. The researchers validated their approach using several grid world examples, a common testbed in reinforcement learning research. By automating the inference of these task blueprints, the method could significantly reduce the engineering overhead required to deploy robots for intricate, long-horizon tasks where reward specification is notoriously difficult.