Stochastic Ray Tracing for the Reconstruction of 3D Gaussian Splatting

A research team including Peiyu Xu and Xin Sun has introduced a novel, differentiable framework called Stochastic Ray Tracing for 3D Gaussian Splatting (3DGS). This method directly tackles the primary speed bottleneck of existing ray-tracing approaches for 3DGS: the need to sort every Gaussian that intersects a camera ray. By employing an unbiased Monte Carlo estimator, their technique randomly samples only a small subset of Gaussians per ray to compute pixel-color gradients, completely bypassing the expensive sorting step. This allows it to match the reconstruction quality and speed of traditional, faster rasterization-based 3DGS while substantially outperforming older sorting-based ray-tracing methods.

For standard 3DGS scenes, this means achieving photorealistic quality without the traditional limitations of rasterization, such as rigid pinhole camera assumptions. More significantly, the same stochastic engine powers a breakthrough for relightable 3DGS—scenes where lighting can be changed after capture. Instead of relying on approximations like shadow maps, the framework can perform fully ray-traced shadow calculations for each Gaussian, delivering notably higher reconstruction fidelity than prior work. This finally delivers on the full generality that ray tracing promises for 3DGS, enabling accurate simulations of complex light interactions like reflections, refractions, and soft shadows that were previously impractical or inaccurate.