Prompt-Guided Prefiltering for VLM Image Compression

Researchers Bardia Azizian and Ivan V. Bajic have introduced a novel method called 'Prompt-Guided Prefiltering' to tackle the inefficiency of sending full-resolution images to cloud-based Vision-Language Models (VLMs) like those used for visual question answering. Traditional image codecs and even newer Image Coding for Machines (ICM) methods are suboptimal for VLMs because they either preserve human-centric details or assume a fixed task, failing to adapt to the open-ended, prompt-driven nature of modern AI. This creates unnecessary bandwidth costs when query images are uploaded for processing.

Their solution is a lightweight, plug-and-play module that acts as a smart filter before compression. It takes the user's text prompt (e.g., 'How many people are in this room?') and uses it to identify which regions of the image are most relevant to answering that specific query. The module then preserves critical details in those areas while smoothing out or simplifying 'task-irrelevant' background elements. This pre-processed image is much easier to compress, leading to massive bandwidth savings.

The key breakthrough is the system's adaptability and compatibility. It is 'codec-agnostic,' meaning it can work with any existing encoder, conventional or AI-based. In experiments across several VQA benchmarks, the approach delivered an impressive 25-50% reduction in average bitrate (the amount of data needed) without sacrificing the VLM's accuracy on the intended task. This directly translates to lower cloud compute and data transfer costs for companies deploying VLMs at scale. The paper has been accepted to IEEE ICME 2026, and the source code is publicly available, paving the way for integration into real-world AI pipelines.