Absolute abstraction: a renormalisation group approach

A team of researchers including Carlo Orientale Caputo, Elias Seiffert, and Matteo Marsili has published a groundbreaking paper titled 'Absolute abstraction: a renormalisation group approach' that challenges fundamental assumptions about how AI systems develop abstract representations. The paper argues that while neural network depth helps create abstraction by combining lower-level features (like edges into shapes), depth alone is insufficient for developing truly abstract representations. Instead, the researchers propose that the breadth of training data is equally critical, using a renormalisation group approach—a mathematical framework borrowed from physics—to show how representations expand to encompass broader datasets.

The researchers identify a unique fixed point in this transformation called the 'Hierarchical Feature Model,' which they propose as a candidate for absolutely abstract representations. They tested this theoretical framework through numerical experiments using Deep Belief Networks and auto-encoders trained on datasets of varying breadth. The experiments confirmed that neural network representations approach the Hierarchical Feature Model as both data breadth increases and network depth grows, validating their theoretical predictions. This work bridges theoretical physics concepts with practical machine learning, suggesting that future AI systems might require both architectural depth and exceptionally diverse training data to achieve human-like abstraction capabilities.