HistoFID methodology standardizes pathology AI evaluation, slashing score variability by 89%

Fréchet Inception Distance (FID) is a widely used metric for comparing real and generated image distributions, but in digital pathology it suffers from a critical flaw: the choice of encoder (e.g., Inception-v3 vs. histology-specific foundation models) can change the raw score by up to 30-fold for the same tile sets. A new paper from Bhat et al. (arXiv:2607.20584) systematically demonstrates this inconsistency across six popular encoders—Inception-v3, Phikon-v2, CONCH, UNI2-h, Virchow2, and Prov-GigaPath—using an in-house cohort of ~500,000 H&E and IHC tiles from 2,119 slides plus 100 TCGA-BRCA slides. They show that raw scores cannot be compared without naming the encoder, and that the ordering of models does not follow embedding dimension.

The authors propose HistoFID, a simple normalization: express each Fréchet distance as a ratio to the encoder's own within-cohort baseline (the distance between two halves of the same cohort). This reduces the across-encoder coefficient of variation by ~89% within a cohort and ~58% across cohorts. The encoders naturally separate into a 'sensitive' group (CONCH, Phikon-v2, Inception-v3) and an 'invariant' group (UNI2-h, Virchow2, Prov-GigaPath), and crucially, this split can reverse which generative model is judged more realistic. At the slide level, using attention-pooling encoders dramatically increases the distance (up to 320-fold) by capturing per-slide composition—information lost in pooled patch distances. Finally, the paper evaluates TuroCompress, a proprietary pathology codec, finding it achieves the highest reconstruction fidelity at the smallest file size among tested codecs. The authors release the normalization protocol, per-encoder perturbation panels, and feature extracts to enable reproducible evaluation.