Johns Hopkins' new AI achieves 3.7mm depth accuracy for real-time surgical guidance

Depth perception is critical for autonomous and semi-autonomous robotic laparoscopic surgery, but conventional fringe projection profilometry requires multi-shot acquisition, digital-micromirror-device (DMD) projectors, and precise projector-camera synchronization—hard to integrate into compact laparoscopes. Researchers from Johns Hopkins University (Rodgers, Le, Jang, Wei, Opfermann, Kam, Krieger, Kang) propose a synchronization-free alternative: a passive LED-illuminated binary mask projected through one channel of a dual-channel laparoscope, while the second channel captures the fringe-illuminated target. A VQ-VAE (vector-quantized variational autoencoder) encodes each input into discrete latents, and a latent-space U-Net predicts depth directly, eliminating the need for a separate mask-prediction branch.

Trained on 722 paired phantom images with Zivid 3D camera reference depth reprojected into the SSLE image frame, the model achieves an MAE of 3.70 mm, AbsRel of 0.0326, delta-1 accuracy of 0.962, and delta-1² accuracy of 0.970. This outperforms the dual U-Net MaskNet + DepthNet baseline and off-the-shelf monocular depth models. The pipeline runs at 26.0 Hz over 301 consecutive frames on an NVIDIA A100 GPU, enabling real-time video-rate depth estimation. Notably, it reconstructs depth without an explicit segmentation stage, though authors stress that dataset size and SSLE-Zivid calibration accuracy are critical to performance. The work, posted on arXiv (2608.05109), demonstrates a practical path toward millimeter-accurate, sync-free 3D guidance for robotic surgery.