MIT study reveals how DNNs generalize to rotated objects

A new study published in Transactions on Machine Learning Research (TMLR 2025) by Avi Cooper and colleagues from MIT, Harvard, and other institutions investigates how Deep Neural Networks (DNNs) recognize objects in orientations they've never seen during training. Using the paper arXiv:2109.13445, the researchers demonstrate that DNNs achieve this generalization by 'disseminating' orientation-invariance learned from familiar objects that were observed from many viewpoints. This capability strengthens as the model is trained on more familiar objects, but only for 2D rotations—like in-plane rotations of an image—not for full 3D rotations or novel depth perspectives.

The team identified a neural mechanism behind this behavior: specific neurons become tuned to features that are common between familiar and unfamiliar objects, enabling the network to apply learned rotational invariance to new categories. This suggests that deep learning models can develop brain-like generalization strategies without explicit 3D structure or separate viewpoint normalization. The findings have practical implications for computer vision systems that must handle real-world variability—such as autonomous driving, robotics, and object recognition in cluttered environments—where objects frequently appear in unusual angles. By understanding these emergent mechanisms, researchers can design more sample-efficient training regimes that mimic this capability, potentially reducing the need for massive, viewpoint-augmented datasets. The study also opens a window into how biological visual systems might achieve similar feats, bridging machine learning and neuroscience.