Differentiable RL teaches fish-like robot to track paths
New method trains agile fish-like robots in simulation, then works perfectly in real water.
Bioinspired fish-like robots have struggled with control due to complex fluid-structure interactions and nonlinear underactuated dynamics. Now, researchers from what appears to be a US university have built a computationally efficient simulation platform that approximates the robot’s motion, enabling the use of reinforcement learning—previously limited for such systems. They treat the PID controller gains as learnable parameters, optimized via backpropagation through time and a curriculum training schedule. This differentiable RL approach allows the robot to learn effective path tracking policies entirely in simulation.
When transferred to the physical fish-like robot, the learned policy demonstrated an excellent match to simulation results, proving the robustness of the training pipeline. The work, accepted to IROS 2026, opens the door to more agile and adaptive underwater robots. By combining differentiable simulation with RL, it bridges the sim-to-real gap for a class of robots that has historically been difficult to control autonomously.
- Uses backpropagation through time with differentiable RL to learn PID gains for path tracking.
- Simulation platform accurately approximates fluid-structure interaction for computational efficiency.
- Policy transferred to physical robot showed excellent match to simulation, enabling robust real-world control.
Why It Matters
Brings bioinspired underwater robots closer to real-world deployment with efficient sim-to-real training.