CO-Calib boosts multi-fisheye calibration success from 68% to 99%

Researchers at HKUST have published a paper titled 'Observation Quality Matters: Robust Multi-Fisheye Calibration via Failure-Oriented Analysis' (arXiv:2607.05777) that tackles a persistent problem in robotics: calibrating multi-camera systems with wide-angle fisheye lenses. The team, led by Peize Liu, conducted a rigorous failure-oriented analysis and discovered that the primary cause of calibration breakdown is not detector recall loss or poor image-point distribution, but rather intrinsic initialization failures. Specifically, observations with limited radial span couple focal scale with fisheye projection-shape parameters, producing ill-conditioned updates in the solver.

To address this, the authors propose CO-Calib, a plug-in framework that works alongside existing calibration pipelines without modifying the optimization backend. CO-Calib includes a robust learning-based target detector and an error-analysis-guided frame selector that constructs initialization-friendly anchors, ensures co-visible multi-camera constraints, and adds coverage-completion frames. In extensive tests on both synthetic and real multi-fisheye setups, CO-Calib raised the overall calibration success rate from 68.1% to 99.3%, while also improving extrinsic accuracy and real-world stability. The code is publicly available, making this a practical advancement for robotics, autonomous driving, and VR/AR applications that rely on multi-camera systems.