Shallow PNNs over finite fields: new expressivity bounds

A new arXiv paper by Maksym Zubkov, Carol Wu, Shiwei Yang, Param Mody, and Yifei Chen (submitted July 2026) tackles a fundamental question: how expressive are shallow polynomial neural networks when the weights and activations live in a finite field? The authors focus on shallow PNNs—networks with one hidden layer—using monomial activation functions. For a given architecture (input dimension, hidden width, output dimension, polynomial degree), they define a 'neuromanifold' as the image of the weight-to-function map inside a product of polynomial rings. Expressivity is then measured by the cardinality (number of distinct functions) of this neuromanifold.

The team derives natural lower and upper bounds on this cardinality, turning the problem into counting rational points on algebraic varieties over finite fields—a topic deeply connected to the Weil conjectures. More strikingly, they present a specific architecture where the neuromanifold’s size (and structure) changes drastically when moving from a field of characteristic zero (e.g., real numbers) to a finite characteristic field (e.g., GF(p)). This result shows that the arithmetic of the underlying field is not merely a technical distraction; it can fundamentally alter what functions a fixed network architecture can learn. The work has implications for neural networks deployed in cryptographic settings or on low-precision hardware where operations are performed over finite fields.