VideoOdyssey benchmark tests AI on 109-minute videos with continuous reasoning

Real-world long video understanding demands continuous tracking, information integration, and memory retention over extreme durations—a cognitive load most benchmarks fail to capture. Existing tests often use short, isolated clips, missing the challenge of ultra-long-context reasoning. To address this, researchers introduce VideoOdyssey, built around a core metric: continuous certificate length, i.e., the video segment a human must watch to answer a question definitively.

VideoOdyssey includes two subsets: VideoOdyssey-V for pure visual understanding and VideoOdyssey-AV for synchronized audio-visual comprehension. Videos average 109 minutes—spanning documentaries, tutorials, sports, and more—requiring models to reason across 5 granular levels (seconds to hours). Extensive evaluations of current multimodal LLMs reveal they struggle not only with simple retrieval but also with continuous reasoning across varying contexts, fine-grained perception, and non-verbal omni-modal understanding, highlighting fundamental bottlenecks in long-video AI.