Subjective Portrait Region Cropping in Landscape Videos with Temporal Annotation Smoothing

Researchers from the University of Texas at Austin and Google have released the LIVE-YT VC database, the largest publicly-available subjective video portrait region cropping dataset, containing 1,800 videos annotated by 90 human subjects. Sourced from the YouTube-UGC and LSVQ databases, this resource addresses a critical gap in enabling AI to intelligently crop landscape videos to portrait or other aspect ratios without distorting content or losing meaning. The team also introduced LIVE-YT VC++, a post-processed version that applies a novel intra-frame temporal filter to smooth subjective annotations across frames, reducing noise in human labeling.

To validate the dataset, the researchers deployed the SmartVidCrop algorithm and fine-tuned state-of-the-art video grounding models for aspect ratio change tasks, demonstrating significant improvements in preserving visual quality and semantic intent. The labels bear similarities to video saliency annotations, prompting additional analysis to explore their overlap. The project is under review at IEEE Transactions on Image Processing, and the code, models, and dataset will be open-sourced, providing a benchmark for future research in mobile-friendly video transformation.