BrickAnything generates stable Lego-like brick structures from any 3D shape

BrickAnything, developed by Zhengyang Ni and colleagues, tackles the challenge of converting arbitrary 3D shapes into physically buildable brick structures (like Lego). Unlike prior methods that rely on heuristic optimization (which fails when shapes don't fit predefined constraints) or generate brick sequences without modeling geometry, BrickAnything uses point clouds as a unified input and predicts brick sequences via an autoregressive framework. The key innovation is a structure-aware tree tokenization that captures local attachment relations between bricks, making generation more aligned with real construction. This reduces invalid intermediate states compared to conventional ordering strategies.

To further ensure buildability, the team introduces three post-training techniques: preference-based alignment to optimize for stability, validity-constrained decoding to avoid illegal placements, and adaptive rollback to correct errors. Extensive experiments show BrickAnything produces geometrically faithful and physically realizable structures while significantly lowering rollback and regeneration needs. The work has implications for automated design, robotics assembly, and educational kits, bridging AI and physical construction.