Intrinsic-GS segments 4D scenes without masks, 12.5x faster

Intrinsic-GS introduces a paradigm shift for segmenting dynamic 3D scenes represented by 4D Gaussian Splatting. Current pipelines rely on generating 2D masks from large foundation models (e.g., SAM) across multiple frames and views, then lifting them into the Gaussian representation—a costly process sensitive to mask quality. Instead, Intrinsic-GS asks how much object-level structure can be recovered directly from the Gaussians. It constructs a sparse affinity graph using intrinsic cues: appearance (RGB), orientation (covariance rotation), scale (covariance eigenvalues), deformation trajectories (per-Gaussian motion over time), and rendered-boundary cues (edge information from a non-learned Sobel filter on rendered views). The graph is partitioned via Leiden community detection, requiring no external masks or learned feature fields.

On standard benchmarks, Intrinsic-GS achieves strong segmentation: 0.746 mean IoU on Neu3D (dynamic indoor scenes) and 0.575 on HyperNeRF (complex deformations). Remarkably, a geometry-only variant (using only orientation, scale, and deformation cues) reaches 0.902 mIoU on Neu3D, matching SAM-supervised TRASE. On HyperNeRF, Intrinsic-GS runs 12.5x faster than than mask-generation and feature-rendering stages of supervised pipelines. These results demonstrate that Gaussian representations inherently encode object-level information, offering a fast, mask-free path to segmentation—especially valuable when external masks are unreliable or expensive (e.g., medical imaging, long video sequences). The method is training-free, pipeline-agnostic, and points toward more robust, generalizable 4D scene understanding.