Novel Fourier embeddings smooth angle representation for ML models

A new paper from Jakeb Chouinard tackles a classic problem in machine learning: how to represent periodic signals like angles and directions without the boundary discontinuities that plague scalar representations like radians or degrees. The core insight is that nearby angles near the pi wrap-around point (e.g., -179° and +179°) appear far apart in linear space, causing errors in tasks like orientation estimation, navigation, and physics simulation. The solution uses Fourier-based embeddings projected into high-dimensional spaces—specifically, Spatial Semantic Pointers (SSPs)—to create smooth, periodic representations where cosine similarity naturally respects angular distance.

The work formalizes two important kernel families using this framework: the Dirichlet kernel and the periodic Gaussian (also known as the von Mises) kernel. By constructing these kernels as dot products of SSP vectors, the approach ensures that distant angles produce near-zero similarity while nearby angles remain highly similar, regardless of the 2π boundary. This neurally-plausible representation scheme can be integrated into neural networks, attention mechanisms, and graph models. The paper emphasizes that these embeddings allow fine-grained control over kernel shape, enabling tasks like rotary position encoding (RoPE) alternatives and improved spatial reasoning. While experimental benchmarks are not included in this theoretical paper, the formalism lays groundwork for better handling of periodic data in AI systems, from autonomous driving to weather forecasting.