Analysis of the Geometric Structure of Neural Networks and Neural ODEs via Morse Functions

A new mathematical analysis using Morse functions proves that neural networks and Neural ODEs have inherent architectural flaws affecting their ability to learn. The research shows critical points—essential for learning—cannot exist if hidden layers shrink or if the phase space dimension is too small. For architectures where critical points do exist, they are almost always non-degenerate, which impacts universal approximation capabilities. The findings provide formal constraints for designing more effective and stable AI models.