HoloLens 2 AR system replaces surgical cutting guides with 4mm accuracy

Mandibular reconstruction after segmental resection traditionally relies on patient-specific cutting guides 3D-printed from CT scans. These physical guides add cost and lead time, cannot adapt intraoperatively, and risk interrupting surgery if sterility is compromised. Researchers Yue Yang and Jie Ying Wu propose a markerless augmented reality alternative that registers a virtual cutting guide directly onto the exposed mandible using a HoloLens 2 time-of-flight camera. Their method extends surface-based registration to the transoral setting, where teeth form the most distinctive visible geometry. After camera calibration, the user provides only a rough head-based alignment to crop the region of interest. A teeth-weighted global stage computes correspondences and solves a truncated least-squares rigid alignment, followed by an asymmetric point-to-plane ICP stage that refines the complete CT mandible against the partial depth cloud. The virtual guide is then placed in the HoloLens world frame, with pose updates and interpolation tracking target motion.

The researchers validated the approach with a blinded phantom protocol using 30 target registration error points under full, intermediate, and teeth-only exposure conditions. Median TRE was 4.05 mm, 6.10 mm, and 7.10 mm respectively, with a motion-to-display latency of 0.805 seconds. These results suggest AR-based guidance is approaching clinical viability for mandibular reconstruction. The workflow removes mounted fiducials and manual landmark selection entirely, instead relying on surface geometry that is naturally present. While the errors are higher than the sub-millimeter accuracy typical of physical guides, the ability to eliminate printed hardware could reduce costs, shorten preoperative lead time, and allow intraoperative plan adjustments. The method offers a testable path toward transoral cutting guidance, potentially transforming how surgeons execute CT-based plans. Further work is needed to improve accuracy under reduced visibility, but the feasibility demonstrated here marks a significant step toward replacing physical prints in craniomaxillofacial surgery.