Stanford team’s depth AI boosts surgical robot precision

Stanford researchers developed a contact detection system for robot-assisted surgery using only depth data from stereo cameras, bypassing the limitations of RGB-based approaches. Their method processes depth frames to identify minimum-distance patches around surgical tools, converting spatial data into a scalar signal ($-\log_{10}|d|$) that correlates with actual contact. By training a two-state hidden Markov model on six palpation sessions and testing across four unseen scenarios—including different tasks and phantoms—the system achieved a held-out macro F1 score of 0.927 and AUPRC of 0.980.

Compared to a reproduced RGB-based contact classifier from prior work—which excelled only in identical scenarios (F1 0.965) but failed in varied conditions (macro F1 0.320)—the depth-based model demonstrated superior transferability. The team’s approach leverages stereo depth’s physical cues over visual features, making it robust to changes in lighting, tissue appearance, or surgical tasks. This work could directly improve safety and control in autonomous surgical robots by providing reliable contact feedback without relying on error-prone RGB models.