JK-EGW: New optimal transport method aligns multimodal embeddings with scarce data

A new paper from researchers Yixuan Florence Wu, Yilun Zhu, and Naichen Shi proposes JK-EGW, a framework for aligning multiple modalities into a shared latent space without relying on dense paired data. The method builds on Gromov-Wasserstein optimal transport, but replaces raw feature distances with a global affinity kernel that captures fine-grained similarities both within and across modalities. This structure-preserving approach gives users explicit control over the latent embedding's geometry and distribution, addressing a common pain point in multimodal learning where strong pretrained encoders exist but paired cross-modal data is expensive to collect.

Theoretical analysis shows JK-EGW achieves a parametric sample complexity rate of n^{-1/2}, matching the standard rates for entropic and Gromov-Wasserstein optimal transport. Algorithmically, the authors derive a scalable alternating procedure that uses a low-rank kernel approximation and variational lifting to reduce the quadratic objective to efficient entropic optimal transport updates, leveraging existing solver infrastructure. Empirically, they evaluate post-hoc alignment of embeddings from pretrained encoders in data-scarce regimes, where JK-EGW consistently outperforms current baselines on multimodal retrieval tasks. For professionals, this suggests a practical path to better cross-modal search and representation learning when annotated data is scarce.