Pretrained vision models enable zero-training obstacle avoidance for robots

A team of researchers (Jagnandan et al.) from the paper 'Time-to-Collision Based Dynamic Obstacle Avoidance Using Pretrained Vision Models for Robots in Unstructured Environments' tackles a key robotics challenge: avoiding moving obstacles in outdoor settings without relying on expensive training data or simulation-to-real transfer. Their pipeline leverages UniDepth, a large pretrained monocular depth estimation model, to generate dense depth maps from a single RGB camera feed—eliminating the need for stereo cameras or LiDAR at inference time. To track dynamic obstacles, they extend the SuperPoint and SuperGlue feature correspondence pipeline across long frame sequences, projecting 2D keypoints into 3D using camera intrinsics and predicted depth. Bundle adjustment refines the trajectories, and per-keypoint time-to-collision (TTC) is computed. A simple 2D motion primitive in the ground plane then moves the robot away from the closest point of approach of the minimum-TTC keypoint.

Evaluated on real-world data from the M3ED dataset, the method achieves a precision of 0.49 and recall of 0.38 in identifying frames where TTC is below 1 second. Critically, it generates the correct evasive motion direction 84% of the time for true positive detections and detects at least one imminent-collision frame for 20 out of 22 unique physical obstacles. Unlike end-to-end learned systems that require thousands of hours of robot-specific data, this approach eliminates model training entirely—only 74 seconds of data are needed for hyperparameter tuning. This demonstrates exceptional data efficiency while preserving interpretable, generalizable behavior across diverse obstacle types (e.g., pedestrians, vehicles). The work is particularly relevant for autonomous mobile robots operating in unstructured environments where large-scale training data is impractical.