Hybrid transformers beat CNNs and vision-language models in retinal screening study

A new study systematically benchmarks 12 deep learning architectures across four model families—CNNs, vision transformers, hybrid CNN-transformers, and vision-language models—for multi-disease retinal screening. Using the Retinal Fundus Multi-disease Image Dataset (RFMiD) covering 28 disease classes, the researchers evaluated binary screening (any disease vs. normal) and multi-label classification. All models achieved AUC above 84% for binary screening, but attention-based models—specifically SwinTiny, CoAtNet0, and MaxViTTiny—delivered the highest performance. Hybrid CNN-transformer backbones also excelled in macro and micro F1 scores for multi-label classification. Vision-language models such as CLIP ViT-B/16 and SigLIP-Base384 were competitive with CNN baselines but fell short of the top transformers and hybrids.

External validation on the Messidor-2 dataset for referable diabetic retinopathy confirmed the trend: hybrid and transformer models again led, with AUC ranging from 66.8% to 84.7%. The authors emphasize that standardized training and calibration protocols make these results reproducible for real-world deployment. The findings offer clear guidance for selecting AI architectures in automated retinal screening tools, especially in multi-disease settings where accuracy across many conditions is critical. This work, accepted at ICMHI 2026, moves the field closer to clinically viable AI that can handle both common and rare retinal diseases under domain shift.