SCISSR: Scribble-Conditioned Interactive Surgical Segmentation and Refinement

A research team including Haonan Ping and Jian Jiang has introduced SCISSR (Scribble-Conditioned Interactive Surgical Segmentation and Refinement), a novel framework that enables surgeons to segment tissues and surgical instruments using simple freehand scribbles rather than precise points or boxes. The system addresses key challenges in surgical computer vision—irregular shapes, thin structures, specular reflections, and frequent occlusions—by allowing users to draw rough strokes over target areas, which the AI then converts into precise segmentation masks. Built on the foundation of Meta's Segment Anything Model (SAM 2), SCISSR adds only lightweight components including a Scribble Encoder, Spatial Gated Fusion module, and LoRA adapters, all of which interact with the frozen backbone model through standard embedding interfaces to preserve its pre-trained capabilities.

This architectural choice makes SCISSR remarkably flexible; the same components can transfer to other prompt-driven segmentation architectures like SAM 3 without structural modification. In testing, the system demonstrated strong performance both in-domain and out-of-distribution, achieving 95.41% Dice score on the EndoVis 2018 benchmark with just five interaction rounds and 96.30% on the CholecSeg8k dataset with three rounds, outperforming traditional iterative point prompting methods. The framework's scribble-based approach provides a more intuitive and efficient interface for surgical teams compared to existing prompt types, potentially reducing annotation time and improving real-time assistance during complex procedures.