SplatStream enables adaptive 3D scene streaming with fine granular layers
SplatStream adapts Gaussian splats to variable bandwidth for low-latency 3D streaming.
Streaming dynamic 3D scenes in real-time is challenging due to large representation sizes and frame redundancy. Gaussian Splatting (GS) enables high-quality rendering but lacks efficient streaming support. SplatStream addresses this with a scalable framework that decomposes GS scenes into quality and resolution layers, using inter-layer predictive coding to reduce redundancy. It introduces B-frames for temporal quality scalability and a lightweight cross-layer transformer-based predictor for both cross-layer and temporal predictions.
The framework further optimizes packetization using a volume-opacity based importance measure, allowing visually important Gaussian primitives to be transmitted first for progressive refinement. The final scalable bitstream maps to an MPEG-DASH compatible sub-representation structure, enabling fine-grained adaptive, low-latency delivery under varying bandwidth conditions. This approach maintains high visual quality while dynamically adjusting to network constraints.
Accepted at Asilomar Conference 2026, SplatStream paves the way for streaming 3D content in VR/AR and metaverse applications, where bandwidth fluctuations are common. By separating rendering quality from network capacity, it ensures smooth user experiences even on unreliable connections.
- Decomposes Gaussian splatting scenes into quality and resolution layers with inter-layer predictive coding
- Introduces B-frames for temporal quality scalability and a lightweight cross-layer transformer predictor
- Uses volume-opacity importance measure for prioritized packetization and maps to MPEG-DASH for adaptive streaming
Why It Matters
Enables adaptive low-latency streaming of 3D scenes, crucial for VR/AR and metaverse on bandwidth-constrained networks.