Polynomial KANs crack Game of Life where ReLU networks fail

Tashin Ahmed and Q. Tyrell Davis have published a new paper on arXiv (2606.23587) that challenges the default choice of ReLU activation functions and large network scales for learning complex dynamics. Using Conway's Game of Life as a testbed, they demonstrate that minimal neural networks can reliably learn the cellular automaton's rules when equipped with a 2nd degree polynomial activation function, specifically in polynomial Kolmogorov-Arnold Networks (KANs). This is a striking contrast to traditional ReLU networks, which typically need to be oversized to contain lucky 'winning ticket' sub-networks that happen to learn the task.

The authors reframe the problem from a search perspective to a genuine learning problem, arguing that with fitted inductive biases, minimal networks should suffice. Their experiments show that several alternative activation functions outperform ReLU, with the polynomial variant consistently learning Game of Life dynamics—even without optimizing neural weights. This result highlights the importance of matching network architecture to the task at hand, and suggests that scaling up models is not always the answer. The paper, accepted for the 2026 Artificial Life Conference, advocates using cellular automata like Game of Life as simple test domains for developing strategies in physics-based deep learning, interpretable ML, and scientific discovery.