CATRF slashes volumetric video bandwidth using standard codecs

Volumetric media for free-viewpoint video streaming is held back by massive bandwidth demands. Implicit and hybrid representations reduce model sizes, but still require careful compression to reach bitrates comparable to 2D video. Now, a team from UMass Amherst introduces CATRF (Codec-Adaptive TriPlane Radiance Fields), a novel compression framework that puts standard video codecs directly inside the training loop.

CATRF works by quantizing and packing 2D feature planes into codec-friendly canvases, running a standard codec roundtrip (JPEG, VP9, HEVC, or AV1), then unpacking and dequantizing before volume rendering. A straight-through estimator (STE) lets the non-differentiable codec pipeline participate in training, so radiance-field features adapt to real client-side codec distortions without adding any learned codec parameters. On both static and dynamic benchmarks, CATRF achieves better rate-distortion trade-offs than codec-agnostic and learned-codec-in-the-loop baselines, and outperforms recent compressed 3DGS methods in compression efficiency and decoding speed. This work highlights a practical path to low-bitrate, compression-resilient volumetric representations for free-viewpoint video streaming.