FAST-MEL links multimodal entities 1000x faster with 10x less storage

Multimodal entity linking (MEL) is the problem of matching textual and visual mentions of entities (say, a photo of a landmark with a caption) to the correct entry in a knowledge base. Until now, systems either sacrificed accuracy, speed, or storage. FAST-MEL, a new system proposed by Thomas Derrien, Laurent Amsaleg, and Pascale Sébillot (to appear at SIGIR 2026), cracks all three constraints simultaneously.

The researchers built a lightweight encoder-based architecture that represents both textual and visual information for each entity as a single fixed-size vector. This compact representation makes the knowledge base index extremely small — one order of magnitude smaller than even the fastest existing systems. Because matching is done via simple vector similarity, inference runs three orders of magnitude faster than previous state-of-the-art while preserving competitive accuracy. For businesses dealing with millions of media items (news archives, social feeds, e-commerce catalogs), FAST-MEL makes it feasible to link text and image mentions at scale without expensive GPU clusters or massive storage.