KILVO's multimodal odometry keeps humanoid robots stable when sensors fail

Humanoid robots face a unique localization challenge: unlike wheeled robots, they must track complex whole-body motion while walking on uneven terrain, and any sensor dropout can cause instability or a fall. Existing fusion methods often treat sensors equally and fail when one modality degrades. To address this, researchers from the Harbin Institute of Technology (Gao, Liu, Zhang, and Zha) developed KILVO, a kinematic-inertial-LiDAR-visual odometry framework published in IEEE/ASME Transactions on Mechatronics. KILVO exploits all the sensors commonly found on humanoid robots—joint encoders, an IMU, a LiDAR, and a camera—by fusing them in a single error-state iterated Kalman filter (ESIKF) that processes measurements asynchronously to match each sensor's natural rate.

The architecture is deliberately hierarchical: inertial data drives high-frequency prediction, leg kinematics provide proprioceptive constraints at high rate, LiDAR point cloud registration supplies geometric priors, and visual photometric errors refine the estimate. This asynchronous-sequential design prevents fast sensors from being bottlenecked by slower ones and lets each modality contribute where it is strongest. The framework also includes a lightweight contact estimation module that shares information with the state estimator without requiring extra hardware, improving gait awareness on slopes and stairs. In experiments across public datasets and multiple real humanoid robots with different gaits, KILVO demonstrated highly competitive accuracy and output rates, crucially maintaining stable estimates even when LiDAR or camera data was degraded or dropped. The authors have released code and datasets on GitHub, allowing other robotics teams to adopt the system and push humanoid autonomy forward.