New CMDR-Embed model understands document context across pages

A team of researchers (Ryota Tanaka, Taku Hasegawa, Kyosuke Nishida) has published a paper introducing CMDR (Contextual Multimodal Document Retrieval) and a corresponding benchmark, CMDR-Bench. Traditional multimodal document retrieval methods evaluate simple lexical or semantic matching and encode pages independently, ignoring the contextual flow between pages. This limitation means they cannot resolve queries that aggregate information across multiple pages — for example, finding a data table that references a chart from a previous page. CMDR solves this by explicitly modeling document context.

The core contribution is CMDR-Embed, a contextual multimodal embedding framework that jointly encodes multiple pages and derives each page's embedding from a shared contextual representation. To train it effectively, the authors propose CMCL (Contextual Multimodal Contrastive Learning), a loss function that balances contextual modeling with the ability to discriminate individual pages. Experiments show CMDR-Embed significantly outperforms non-contextual embeddings, highlighting the critical role of context in multimodal retrieval. This work, accepted at ECCV 2026, opens the door to smarter search and analysis of complex multi-page documents.