SAGFormer allocates Gaussians adaptively for sharper 3D occupancy predictions

Researchers at Harbin Institute of Technology introduce SAGFormer (Semantic Gaussian Allocation Transformer), a novel method for 3D semantic occupancy prediction. Current semantic 3D Gaussian representations build compact voxel volumes by rendering primitives under voxel-wise supervision. While recent work has improved primitive shapes, initialization, and densification, none explicitly addresses how to choose the most valuable Gaussians when total count is capped to control memory and compute. This causes an allocation bottleneck: simple regions get redundant Gaussians, while complex regions lack needed semantic support. SAGFormer solves this by using a transformer that scores each candidate based on its attributes and local geometric-semantic features, then selects a fixed final set of Gaussians.

Experiments on nuScenes-SurroundOcc and SSCBench-KITTI-360 show that SAGFormer improves occupancy prediction quality under similar final Gaussian counts and raw coverage. The method reduces semantic mixing across class boundaries, strengthens class-consistent voxel support, and yields far fewer unused Gaussians compared to prior approaches. The work demonstrates that explicit capacity allocation is a powerful complement to existing Gaussian refinement techniques, offering a practical path to more efficient and semantically consistent 3D scene understanding for autonomous driving and robotics.