Report of the 5th PVUW Challenge: Towards More Diverse Modalities in Pixel-Level Understanding
New MeViS-Audio track lets AI segment objects by sound alone...
The 5th Pixel-level Video Understanding in the Wild (PVUW) Challenge, hosted at CVPR 2026, pushes the boundaries of computer vision with three specialized tracks designed for real-world, unconstrained scenarios. The MOSE track targets object tracking in densely cluttered and severely occluded environments, while MeViS-Text focuses on localizing targets using motion-focused linguistic expressions. The groundbreaking addition is the MeViS-Audio track, which pioneers acoustic-driven object segmentation—allowing AI to identify and segment objects based solely on sound cues, a modality previously underexplored in pixel-level understanding.
The challenge introduces previously unreleased, highly challenging datasets that stress-test state-of-the-art models. The report, authored by 43 researchers including Philip Torr, analyzes top-performing multimodal solutions submitted by participants, revealing significant advancements in handling diverse modalities. By combining visual, textual, and now audio inputs, the PVUW 2026 challenge charts promising directions for robust video scene comprehension, moving closer to human-like perception that integrates multiple senses for pixel-level understanding in the wild.
- New MeViS-Audio track enables object segmentation using acoustic signals alone
- MOSE track tackles dense clutter and severe occlusion for object tracking
- MeViS-Text localizes targets via motion-focused linguistic expressions
Why It Matters
Multimodal video understanding advances toward human-like perception by integrating audio, text, and vision.