Autonomous Search for Sparsely Distributed Visual Phenomena through Environmental Context Modeling
A new system uses DINOv2 embeddings to search for sparse targets by analyzing their habitat, not just the target itself.
A team from MIT and Woods Hole Oceanographic Institution has developed a novel AI system that enables autonomous underwater vehicles (AUVs) to find sparsely distributed targets, like specific coral species, far more efficiently. The core innovation is teaching the robot to search not just for the target itself, but for the broader environmental context—the habitat features that tend to co-occur with it. Because these context features (e.g., certain rock textures or algae types) are more common and vary more smoothly across a reef than the rare target, they provide a much richer signal for the robot's planning algorithm to decide where to search next.
The method starts from just a single labeled image. It uses DINOv2, a powerful visual foundation model, to generate patch-level embeddings that allow for one-shot online detection of both the target and its context. The robot's adaptive planner then uses this combined signal to navigate. Validated with real AUV imagery from reefs in St. John, U.S. Virgin Islands, the system proved highly effective in offline simulations. When a target is sparsely distributed, the context-aware strategy can sample 75% of the target population in about half the time an exhaustive, lawnmower-pattern search would take, and it consistently outperforms searches guided only by sparse target detections.
This research, accepted for ICRA 2026, addresses a critical limitation in robotic environmental monitoring: limited battery life. By making search intent-driven rather than exhaustive, it dramatically extends the useful mission range for AUVs tasked with finding rare phenomena, from corals to archaeological artifacts or pollution plumes.
- Uses DINOv2 embeddings for one-shot detection of targets and their environmental context from a single image.
- Samples 75% of a sparse target population in roughly half the time of exhaustive coverage searches.
- Validated with real AUV imagery from coral reefs, outperforming strategies using only direct target detections.
Why It Matters
Dramatically extends the range and effectiveness of autonomous robots for environmental monitoring, conservation, and search missions where targets are rare.