SegFS: Dual-Path Framework Enables 14x Faster Video Segmentation on Mobile

SegFS (Segmenting, Fast and Slow) tackles the challenge of real-time open-vocabulary video instance segmentation (OV-VIS) on mobile devices. The framework uses a dual-path architecture: a slow, object-centric model (inspired by DETR) processes sparse keyframes to generate instance-level representations. These representations are then projected back into the backbone feature space to condition a lightweight fast network. This design decouples multimodal semantic understanding from dense mask prediction, shifting instance propagation from object decoding to feature-space conditioning. The fast branch then efficiently relocalizes and segments instances in subsequent frames without recomputing the heavy semantic model. On standard OV-VIS benchmarks, SegFS achieves competitive segmentation performance while reducing latency by up to 14x compared to MOBIUS, a state-of-the-art mobile-oriented model.

The key innovation lies in the temporal propagation strategy. Instead of running the full open-vocabulary model on every frame, SegFS only applies the computationally expensive semantic understanding on sparse keyframes. The lightweight fast network then leverages the conditioned feature space to track and segment objects across frames with minimal overhead. This makes SegFS particularly suited for real-time applications on resource-constrained devices, such as augmented reality, autonomous navigation, and video analytics. The paper demonstrates that the approach maintains high accuracy while significantly improving efficiency, addressing a critical bottleneck for deploying OV-VIS in practice. Published at ECCV 2026, SegFS represents a practical step toward making advanced video segmentation accessible on mobile platforms.