Deep RL enables snake robots to adapt to changing viscosity

A team of researchers has demonstrated that deep reinforcement learning (DRL) can enable snake-like robots to adapt their undulatory locomotion in real-time to changing fluid viscosities, overcoming the limitations of predefined control methods. Because the robots lack onboard sensors to directly measure fluid properties, the task is formulated as a partially observable Markov decision process. The team used an asymmetric actor-critic framework where a teacher policy, trained with privileged information available only in the physics simulator, distills its knowledge into a student policy that relies solely on proprioceptive sensor data (e.g., joint angles, torques). This approach allows the robot to implicitly infer environmental conditions and adjust its gait accordingly.

In simulations spanning a wide range of dynamic viscosities (from 10⁻⁷ to 10⁻² m²/s), the DRL agent autonomously discovered non-sinusoidal adaptive gaits that significantly improve both propulsion velocity and transport efficiency. These gaits break the inherent performance limits of conventional sinusoidal and kinematic-based controllers, which fail under unpredictable fluid dynamics. The findings establish that implicit environment inference via privileged information distillation is an effective and generalizable strategy for robots operating in unstructured environments. This work has practical implications for snake robots used in medical procedures, underwater exploration, and search-and-rescue missions where fluid conditions vary.