Masked Autoencoder achieves 91.3% accuracy on unsupervised steel defect detection

Shrey Patel's new paper on arXiv presents a Masked Autoencoder (MAE) approach for unsupervised steel surface defect recognition—a critical quality control task where labeled defect data is scarce but unlabeled images are abundant. The method uses a Transformer-based encoder that processes only 25% of randomly sampled image patches; the remaining 75% are masked. A lightweight decoder reconstructs the masked regions, achieving a structural similarity (SSIM) of 0.92 and a mean squared error (MSE) of 0.47. An auxiliary defect localization objective is used during training solely as a learning signal, not for evaluation.

After pretraining, the encoder's features are clustered using UMAP for dimensionality reduction and agglomerative clustering, yielding a Hungarian matched accuracy of 91.3% against six known defect categories. This self-supervised framework eliminates the need for costly manual labeling, making it highly scalable for real-world steel production lines. The work demonstrates that MAEs can learn discriminative visual representations even with extreme masking ratios, opening the door to broader industrial anomaly detection applications where labeled data is a bottleneck.