The 'Grain Blindness' Problem in Multimodal Retrieval — and How ELVA's Ranking-Driven RL Solves It
New RL-based approach improves retrieval by 13.1% on multi-grain queries without reward models.
A team of 11 researchers has proposed ELVA (Exploring Ranking-Driven Universal Multimodal Retrieval), a novel framework that overcomes a critical flaw in contrastive learning for multimodal retrieval. The flaw, termed 'grain blindness,' arises because contrastive methods treat all negatives equally as binary opposites to positives, ignoring the fine-grained semantic information in queries. This limits performance on complex, multi-faceted queries. ELVA addresses this by introducing a ranking-driven approach: negatives are ordered by their similarity to the positive sample, allowing the model to learn distinct grain levels from each negative.
Instead of relying on expensive reward models, ELVA extends Reinforcement Learning with Verifiable Rewards (RLVR) to retrieval tasks, using simple rule-based rewards to optimize both the ranking of negatives and the similarity gap between positives and negatives. To evaluate grain blindness, the authors also introduce MRBench, a new benchmark designed specifically for multi-grain query scenarios. ELVA achieves state-of-the-art results across standard retrieval benchmarks, and its notable 13.1% improvement on MRBench demonstrates its effectiveness. The paper has been accepted at ECCV 2026, signaling strong impact in the multimodal AI community.
- ELVA uses rule-based RL (RLVR) instead of reward models to rank negative samples by similarity, addressing grain blindness in contrastive learning.
- Achieves state-of-the-art on standard multimodal retrieval benchmarks and a 13.1% improvement on the new MRBench multi-grain query benchmark.
- Accepted at ECCV 2026; developed by a team of 11 researchers from academia and industry.
Why It Matters
ELVA enables more precise multimodal search by respecting fine-grained query details, critical for complex retrieval tasks in AI applications.