FairEnc: A Fair Vision-Language Model with Fair Vision and Text Encoders for Glaucoma Detection
New method reduces racial and gender bias in medical AI without sacrificing diagnostic accuracy.
Automated glaucoma detection is critical for preventing vision loss, but AI models often exhibit demographic bias. FairEnc, introduced by researchers Mohamed Elhabebe, Ayman El-Baz, and Qing Liu, tackles this by jointly debiasing both the text and image encoders of a vision-language model. For the textual encoder, an LLM generates synthetic clinical descriptions that vary sensitive attributes (e.g., race, gender) while preserving disease semantics, then a contrastive alignment objective encourages demographic-invariant representations. For the visual encoder, they apply mutual information regularization to reduce statistical dependence between features and demographic groups, combined with multi-discriminator adversarial debiasing.
Comprehensive experiments on the public Harvard-FairVLMed dataset show FairEnc significantly reduces disparity metrics like DPD and DEOdds compared to baselines, while maintaining competitive diagnostic accuracy in zero-shot and linear probing evaluations. Additional tests on the private FairFundus dataset confirm that the fairness gains generalize across domains, modalities, and distribution shifts. The code and synthetic clinical notes are open-sourced. This work represents a practical step toward equitable deployment of AI in ophthalmology and other clinical settings where bias can have serious consequences.
- FairEnc jointly debiases text and vision encoders across four sensitive attributes: race, gender, ethnicity, and language.
- Uses LLM-generated synthetic clinical notes and a dual fairness strategy (mutual information regularization + adversarial debiasing) to reduce statistical dependence on demographics.
- Achieves state-of-the-art fairness on Harvard-FairVLMed (measured by DPD and DEOdds) without sacrificing diagnostic performance, and generalizes to private FairFundus dataset.
Why It Matters
FairEnc could reduce diagnostic disparities in real-world glaucoma screening, advancing equitable AI deployment in clinical ophthalmology.