Häberle & Bölcskei's new theory explains why deep learning works on low-dimensional data

Two researchers, Konstantin Häberle and Helmut Bölcskei, have published a theoretical breakthrough that mathematically explains why deep learning models work so well on real-world data. Their paper, 'Function-Counting Theory for Low-Dimensional Data Structures' (arXiv:2607.01010), builds on Thomas Cover's seminal 1965 function-counting theory, which originally assumed data points are in 'general position' — an assumption that ignores the intrinsic low-dimensional structure common in high-dimensional datasets.

The authors refine this core assumption to explicitly account for data lying on low-dimensional manifolds. They derive new dichotomy counts (the number of ways a classifier can separate points) that reflect the actual geometry of the data, and extend Cover's concepts of separation capacity and generalization to low-dimensional settings. This 49-page theoretical work, spanning machine learning, information theory, and combinatorics, provides a rigorous foundation for understanding why overparameterized neural networks generalize so well — a phenomenon long observed but not fully explained. For ML practitioners, the results could inform better architecture choices and data representation strategies.