New survey unifies progress reward modeling for better robot learning

Robotic learning in dynamic environments suffers from a fundamental problem: traditional terminal success signals only indicate task completion, not whether the robot is progressing, stagnating, or regressing. To address this, researchers have explored progress rewards that provide intermediate feedback. However, the field lacks a shared framework, with diverse observations, goal specifications, output signals, supervision sources, and evaluation protocols. In a new comprehensive survey (arXiv:2607.21655) by Jianshu Zhang and 10 co-authors, the authors present a unified view of progress reward modeling for robotic learning. They organize the field into three connected steps: the interface of a progress model (what information it receives and what signal it produces), the internal methods for constructing that signal (revealing assumptions and mechanisms), and the data and benchmarks used for validation. This framework connects what a progress model is, how it is built, and how its quality is measured, enabling meaningful comparisons across studies.

The survey goes beyond taxonomy by summarizing current limitations and future research directions. One key limitation is the lack of standardized evaluation protocols—many methods use different environments, making apples-to-apples comparisons impossible. Another is the challenge of designing progress signals that are both informative and robust to noise. The authors also discuss scaling progress reward modeling to complex, long-horizon tasks and integrating it with foundation models. By providing a comprehensive map of the field, this survey helps researchers and practitioners identify best practices, avoid redundant efforts, and accelerate the development of more intelligent, adaptive robots. It likely becomes a go-to reference for anyone building robotic learning systems that need fine-grained feedback rather than binary success/failure.