DH-Active: iPhone LiDAR depth recovery 38x faster than DINOv2
iPhone LiDAR depth recovery runs in 1.11ms with 1.4% error – no training needed.
A new paper from Jinwen Wen introduces DH-Active, a lightweight geometry back-end that treats the iPhone's LiDAR scanner as a metric ruler rather than the sole source of depth. Instead of relying on learned neural networks, DH-Active uses near-field LiDAR returns to anchor the metric relative pose via PnP, then triangulates visually trackable points (those without valid depth) under that pose. A critical innovation is a selective abstention mechanism: a parallax/reprojection gate leaves explicit holes wherever geometry is ill-conditioned, avoiding forced estimates and preserving accuracy.
Performance numbers are compelling. The core front end runs at a median latency of 1.11ms on CPU (using OpenCV with 14 threads), which is approximately 38x faster than a DINOv2-L visual branch running on GPU in the same timing setup. Across iPhone captures, TUM RGB-D, and ARKitScenes benchmarks, held-out depth is recovered at 1.4–6.7% median relative error. In a controlled ARKitScenes protocol (using only returns within 2m and independent laser scan ground truth), DH-Active achieves 64.2% scene-median coverage of evaluable far-field candidates at 13.4% scene-median relative error. The key contribution: zero learned parameters, near-millisecond CPU cost, and explicit abstention. The paper also details several failed alternatives (single-frame defocus, ICP over visual-inertial tracks, etc.), reinforcing the approach's novelty.
- Median latency of 1.11ms on CPU (~38x faster than DINOv2-L GPU baseline)
- Depth error of 1.4–6.7% median relative error across benchmarks
- Zero learned parameters; uses selective abstention to avoid ill-conditioned geometry
Why It Matters
Brings fast, accurate sparse depth to consumer devices without neural networks – enabling real-time AR and robotics on iPhones.