CLRNet: Targetless Extrinsic Calibration for Camera, Lidar and 4D Radar Using Deep Learning

A research team from TU Delft and the University of Amsterdam has introduced CLRNet, a novel deep learning framework designed to solve a critical bottleneck in autonomous systems: accurately aligning multiple sensors. Extrinsic calibration—determining the precise position and orientation of sensors like cameras, lidars, and 4D radars relative to each other—is essential for creating a unified, reliable perception system. Traditional methods often require cumbersome physical targets and manual intervention, especially challenging for sparse 4D radar data. CLRNet eliminates this need entirely, offering a targetless, end-to-end solution that can handle joint calibration of all three sensors or any pairwise combination.

CLRNet's architecture is its key innovation, incorporating equirectangular projection, camera-based depth prediction, and additional radar channels into a shared feature space. It uses a loop closure loss to ensure consistency. Tested on the View-of-Delft and Dual-Radar datasets, the model demonstrated a dramatic improvement, cutting both median translational and rotational calibration errors by at least 50% over existing methods. The team also explored the model's ability to transfer learning across different datasets, a crucial step for real-world deployment where sensor setups and environments vary.

The implications for the autonomous vehicle and robotics industries are significant. By automating and drastically improving calibration accuracy, CLRNet reduces development time, increases system reliability, and enables more robust perception in diverse conditions. The researchers have committed to making the code publicly available upon the paper's acceptance, promising to accelerate adoption and further innovation in multi-sensor fusion technologies.