Platonic Transformers achieve equivariance with zero extra cost
New attention mechanism uses Platonic solid symmetries for free geometric reasoning.
Transformers lack built-in geometric symmetry handling, forcing existing equivariant methods to sacrifice efficiency. A new paper from international researchers proposes the Platonic Transformer, which redefines attention relative to reference frames based on Platonic solid symmetry groups. This induces a principled weight-sharing scheme that enforces equivariance to continuous translations and discrete symmetries while keeping the exact architecture and cost of a standard Transformer. The attention is formally equivalent to a dynamic group convolution, learning adaptive geometric filters. This enables a scalable, linear-time convolutional variant. The model achieves competitive performance on vision (CIFAR-10), 3D point clouds (ScanObjectNN), and molecular property prediction (QM9, OMol25) at no extra cost, making geometric deep learning more accessible.
- Platonic solid symmetry groups provide a principled weight-sharing scheme for equivariance without extra compute.
- Method preserves exact standard Transformer architecture and computational cost.
- Competitive performance on CIFAR-10, ScanObjectNN, QM9, and OMol25 benchmarks with a linear-time variant.
Why It Matters
Enables geometric reasoning in Transformers for vision, 3D data, and molecular modeling at zero added cost.