Self-supervised AI beats supervised for artwork classification
DINO and CLIP models improve art classification, enabling VR museum navigation
Deep Dive
A systematic investigation compared supervised and self-supervised backbones for artwork classification and retrieval, focusing on paintings. Using the DINO family and CLIP models, the study found that employing a self-supervised backbone leads to consistent improvements in artwork classification. The work also demonstrates applicability in real-world settings, including VR applications for museum navigation. The paper is presented at IRCDL 2026.
Key Points
- Self-supervised backbones (DINO, CLIP) consistently improve artwork classification over supervised methods
- System evaluated on painting datasets with multiple classification and retrieval strategies
- Real-world application demonstrated in VR museum navigation systems (IRCDL 2026 paper)
Why It Matters
Enables more accurate, scalable art AI without manual labeling – key for museums and VR experiences.