Foresight Residual RL boosts robot assembly to 85.6% success

Vision-Language-Action (VLA) policies provide strong manipulation priors but struggle with long-horizon tasks like tight-tolerance assembly due to poor handoff between subtasks. Standard residual reinforcement learning (RL) over a frozen VLA base policy improves each subtask in isolation but yields little gain when chained, because terminal state quality is uncontrolled.

To address this, the authors introduce Foresight Residual RL. They train a visual foresight predictor that estimates the probability of future subtask success given a terminal state, then use it as a reward multiplier during residual RL training via backward induction. In a three-phase wrench-based nut-tightening assembly (grasp, move-insert, rotate) simulated in Isaac Gym, their method achieves 85.6% full-task success, far outperforming standard subtask residual RL (54.5%) and VLA baselines, without sacrificing per-subtask success. The work highlights that improving long-horizon performance requires shaping which successful states are produced at each sub-task, not merely achieving success.