ViPSAM uses visual prompts to boost liver lesion segmentation in non-contrast CT

Proton therapy planning relies on respiratory-gated non-contrast CT (NCCT) for lesion segmentation, but low lesion-to-background contrast makes accurate delineation difficult. While learning-based methods have shown promise, they often fail on non-contrast images. Inspired by clinical practice where radiologists reference contrast-enhanced MRI to identify lesions on NCCT, a team from Samsung Medical Center developed ViPSAM.

ViPSAM is built on Meta's Segment Anything Model (SAM) and introduces two key components: a visual prompt encoder that extracts guidance features from contrast-enhanced images, and a visual-guided cross-attention module that fuses non-contrast and contrast-enhanced features to improve lesion-relevant representations in low-contrast regions. The mask decoder is adapted in a parameter-efficient manner to leverage these visual prompts. Evaluated on liver lesion segmentation from NCCT scans for proton therapy, ViPSAM outperformed representative U-Net and SAM-based methods, demonstrating that cross-modality visual prompting enables more robust segmentation. The work has been accepted at MICCAI 2026.