OSCS-SupCon: New Method Boosts Contrastive Learning Accuracy by 3.4%

Supervised Contrastive Learning (SupCon) has excelled in computer vision by modeling pairwise relationships among samples, but it suffers from two major flaws: negative-sample dilution from the standard InfoNCE loss, and entanglement of category-relevant (common) and irrelevant (style) features. To address this, Bin Wang and Fadi Dornaika introduce OSCS-SupCon (Orthogonal Sigmoid-based Common and Style Supervised Contrastive Learning). Their framework replaces the traditional InfoNCE loss with a sigmoid-based contrastive loss that includes two learnable parameters—temperature and bias—adaptively modulating pairwise decision boundaries to prevent dilution. Additionally, they enforce orthogonality between common and style feature subspaces via a linear projection with ReLU nonlinearity, drastically reducing feature overlap and improving disentanglement of style-irrelevant representations.

Extensive experiments on six benchmark datasets show OSCS-SupCon consistently outperforms state-of-the-art methods across multiple backbone architectures. Notably, on the fine-grained CUB200-2011 dataset with a ResNet-18 backbone, it achieves a 3.4% improvement in classification accuracy over the previous best method (CS-SupCon). Ablation studies confirm the effectiveness of each component—the sigmoid loss and orthogonality constraints—proving that disentangling common and style features directly boosts robustness and generalization. This work provides a practical drop-in improvement for any supervised contrastive learning pipeline, with implications for more reliable vision systems in domains like medical imaging and autonomous driving.