From Inheritance to Saturation: Disentangling the Evolution of Visual Redundancy for Architecture-Aware MLLM Inference Acceleration

A team of researchers has introduced HalfV, a novel framework designed to dramatically speed up inference for high-resolution Multimodal Large Language Models (MLLMs) like LLaVA and Qwen25-VL. The core innovation is a new understanding of visual redundancy, which they disentangle into two distinct types: universal Intrinsic Visual Redundancy (IVR) and architecture-dependent Secondary Saturation Redundancy (SSR). This discovery, based on analyzing truncated matrix entropy, reveals a universal three-stage inference lifecycle that previous acceleration strategies missed. These older methods, such as token pruning, suffered from severe 'backbone dependency,' meaning they worked well on some architectures (e.g., Vicuna) but caused significant performance drops on others.

HalfV tackles this by first applying a unified pruning strategy to mitigate the universal IVR, then adaptively handling the architecture-specific SSR. This two-step, architecture-aware approach allows it to achieve superior efficiency-performance trade-offs across diverse model backbones. In experiments, HalfV delivered standout results on the challenging Qwen25-VL model, retaining 96.8% of its original performance while achieving a 4.1x reduction in computational FLOPs. This significantly outperforms existing state-of-the-art acceleration baselines and provides a more generalizable solution for deploying powerful, vision-capable AI models in real-time applications where speed and cost are critical.