New cross-model routing boosts video QA accuracy by 1.43 points

Implicit video question answering requires models to infer answers from off-screen events, line-of-sight cues, and cross-shot spatial layout – a hard problem where even frontier video LLMs hit an accuracy ceiling. Researchers observed that conventional self-consistency (majority voting across repeated samples of the same model) can hurt because errors on hard questions are correlated. To break this pattern, they developed a pure inference-time procedure called disagreement-based cross-model routing that requires no labels or training.

The method first triple-samples Gemini 3.1 Pro Preview (a native video model) at temperature zero, exploiting the genuine sample-to-sample variance in its video-processing pipeline. On the ~20% of questions where the three samples disagree, it routes them to a second model – Claude Opus 4.8 – which consumes uniformly sampled frames with adaptive thinking. On the 1001-question validation set, AvgAcc improves by +1.43 over the best single Gemini sample, with per-category gains concentrated on Motion & Trajectory (+5.49), Inferred Counting (+3.45), and Vertical Spatial Reasoning (+1.82). The same pipeline on the held-out 172-question CVPR 2026 ImplicitQA test set achieves 82.03 AvgAcc / 79.71 MacroAvgAcc (+1.81), confirming the result on an independent split.