RedFlow turns robot failures into action-level corrections, lifting success to 74.7%

Flow-matching Vision-Language-Action (VLA) models show promise for robotic manipulation but often stumble during deployment due to distribution shifts, causing compounding errors. Offline reinforcement learning can help refine policies with rollout data, yet existing approaches either discard failure examples or exploit them only at the trajectory level, leading to low learning efficiency and persistent mistakes. RedFlow, from researchers at including Zhengyang Yan and colleagues, instead redirects failures into fine-grained, action-level corrective supervision. Its Context-Aware Corrective Matching identifies which individual actions caused failures and retrieves successful alternatives from similar contexts as corrective targets, while its Adaptive Redirection Objective jointly reinforces successful actions, suppresses undesirable ones, and steers recoverable failures toward those targets. This converts both good and bad experiences into dense supervision, enabling robust recovery from mixed-quality data.

On the LIBERO benchmark and three real-world manipulation tasks, RedFlow outperformed state-of-the-art offline RL baselines, improving real-world success from 56.7% to 74.7%. It also matched strong on-policy methods such as PPO, GRPO, and DDPO while requiring roughly an order of magnitude fewer training samples. This suggests that dense, action-level feedback from failures can dramatically improve both sample efficiency and robustness in VLA robot policies. The paper is available on arXiv (2607.27782) and points toward more reliable sim-to-real transfer for robotic manipulation.