SSCD method lifts unpaired medical AI classification with shared codebook distillation

Cross-modal knowledge distillation lets a strong teacher modality (like OCT) train a cheaper student modality (like fundus images) for medical diagnosis. But in real clinical settings, these modalities are often unpaired—collected from different patient cohorts with incompatible feature spaces—making standard instance-level distillation unreliable. The new SSCD method from researchers Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le, and Hyunseung Choo tackles this by representing each image as a distribution over a shared, modality-agnostic discrete codebook. It aligns these distributions across modalities both globally and class-conditionally, so no paired samples or directly comparable raw features are needed.

The codebook is trained online with exponential moving average updates and kept diverse through entropy regularization and dead-code restart. At inference, the teacher and codebook modules are discarded, leaving only the student encoder and classifier—so deployment stays lightweight. On two heterogeneous settings, OCT-to-fundus retinal disease classification and CT-to-chest-X-ray pneumonia classification, SSCD improved the student from 64.5 to 70.2 macro-F1 and from 73.8 to 76.3 macro-F1, outperforming all distillation baselines evaluated. This approach makes cross-modal medical AI practical even when paired datasets are unavailable, and the released code and pretrained models could accelerate adoption in research and clinical workflows.