WorldBench benchmark exposes MLLM limits with 64% accuracy ceiling

A team of researchers led by Yida Yin and 11 co-authors has released WorldBench, a new benchmark designed to stress-test multimodal large language models (MLLMs) on visual diversity and reasoning. Unlike existing benchmarks that expand task types without broadening visual coverage, WorldBench builds a taxonomy of thousands of visual concepts spanning multiple domains—such as living things, objects, and scenes. The team then curates a wide collection of images from search engines and existing datasets to represent the visual world comprehensively. To ensure difficulty, they use structured trial-and-error to manually design questions that frontier MLLMs consistently fail to answer correctly.

In quantitative and human evaluations, WorldBench achieves higher visual diversity than any existing diverse benchmark. Testing 15 state-of-the-art MLLMs reveals stark limitations: the strongest model reaches only 64.0% accuracy, while several perform only marginally above chance. These results suggest that even advanced multimodal models lack robust visual understanding when faced with varied real-world inputs. The authors hope WorldBench will drive progress by emphasizing visual diversity as a critical dimension in benchmark design, pushing the field toward more capable and reliable multimodal AI systems.