Lookalike3D: Seeing Double in 3D

Researchers Chandan Yeshwanth and Angela Dai have introduced Lookalike3D, a novel AI system that tackles the previously overlooked challenge of detecting repeated objects in 3D environments. The core task is to classify pairs of objects in a scene as identical, similar, or different using only multiview images as input. To power this, the team developed a specialized multiview image transformer that effectively harnesses strong semantic priors extracted from large, pre-trained image foundation models, allowing it to understand object relationships at a deep level.

To train and validate their model, the researchers created the 3DTwins dataset, a substantial new benchmark containing 76,000 manually annotated pairs of objects categorized as identical, similar, or different. This dataset is built upon the extensive ScanNet++ repository of 3D indoor scenes. On this challenging task, Lookalike3D demonstrates a massive 104% improvement in Intersection over Union (IoU) compared to existing baseline methods, showcasing its superior accuracy in distinguishing object lookalikes.

The practical impact of this research is significant for robotics and augmented reality. By turning repeated objects from noise into a powerful structural cue, Lookalike3D enables more consistent and higher-quality 3D scene perception. Specifically, it improves downstream tasks like joint 3D object reconstruction—where understanding that two chairs are identical helps build a better model of 'chair'—and part co-segmentation, where identifying similar parts across objects leads to more coherent scene understanding. The team has committed to releasing their code, dataset, and models publicly, accelerating progress in 3D computer vision.