RLMM-Flow: Latent-space RL boosts robot manipulation beyond imitation

RLMM-Flow tackles the challenge of mobile manipulation—generating coordinated whole-body action chunks that meet goals, avoid collisions, and respect kinematic constraints. Traditional flow-based policies learn multimodel motion priors from demonstrations but cannot exceed demo quality. The authors pretrain a flow policy on expert demos, then freeze it and train a latent steering network to guide initial noise toward higher-value actions using reinforcement learning. A crucial innovation is coarse-to-fine latent steering: it first controls a horizon-shared latent representation, then expands to full-dimensional residual representations, stabilizing high-dimensional latent optimization. An action-space critic is warmed up before joint latent critic and actor training.

Evaluated on mobile manipulation motion-planning benchmarks, RLMM-Flow substantially improves task success and collision avoidance compared to imitation-only flow policies and existing RL post-training methods. It maintains the fast inference speed of flow-based policies. This work demonstrates that combining imitation learning with latent-space RL can effectively push policy quality beyond demonstration distribution without sacrificing inference efficiency, making it a practical approach for real-world robotic systems.