Universal Approximation Theorem for Input-Connected Multilayer Perceptrons
New neural network design adds direct input connections to every hidden layer, enabling universal approximation.
Researcher Vugar Ismailov has published a formal proof establishing the universal approximation capabilities of a novel neural network architecture called the Input-Connected Multilayer Perceptron (IC-MLP). Unlike standard MLPs where information flows strictly from one layer to the next, the IC-MLP introduces direct affine connections from the raw input vector to every hidden neuron in the network. This creates a more densely connected information pathway, allowing each processing unit to reference the original input data directly, not just a transformed version from the previous layer.
In the 19-page paper (arXiv:2601.14026v2), Ismailov first analyzes the architecture in a univariate setting, providing explicit formulas for network functions across an arbitrary number of hidden layers. The core theoretical contribution is a proven universal approximation theorem: a deep IC-MLP with a nonlinear activation function can approximate any continuous function on a compact subset of ℝⁿ. This extends the famous, foundational universal approximation theorems for standard neural networks to this new, more connected architecture. The work provides a rigorous mathematical justification for exploring these types of models, which could lead to networks that learn certain functions more efficiently or with different internal representations than conventional designs.
- Architecture adds direct input-to-hidden neuron connections alongside standard layer links
- Proves universal approximation theorem for continuous functions on compact ℝⁿ subsets
- Requires only a nonlinear activation function, matching a key condition of classic theorems
Why It Matters
Provides a theoretical backbone for designing more expressive and potentially more data-efficient neural network architectures.