Researchers propose STAMPlus for 2.6x faster MLLM segmentation

A team led by Jiazhen Liu has proposed STAMPlus, a breakthrough in multimodal large language model (MLLM) segmentation that resolves the longstanding 'segmentation trilemma' of balancing performance, dialogue ability, and speed. The paper introduces Structured All-Mask Prediction (STAMP), which decouples autoregressive dialogue from non-autoregressive mask prediction using a hybrid attention mechanism. This approach enables simultaneous foreground/background classification in a single pass while preserving the model's multimodal instruction-following capabilities.

The researchers then extend this to STAMPlus, which introduces target IDs and optional bounding boxes to support open-vocabulary semantic, instance-aware, and remote-sensing segmentation. In benchmarks, STAMPlus achieves state-of-the-art performance across settings while reducing inference latency dramatically—from 13.50 seconds to just 5.16 seconds for 12-category segmentation. The model maintains these capabilities through a unified checkpoint, eliminating the need for repeated target-specific predictions that plague existing approaches.