Persistence diagrams of random matrices via Morse theory: universality and a new spectral diagnostic
A new 'persistence entropy' metric outperforms standard tools for analyzing random matrix ensembles with 97.8% AUC.
Matthew Loftus's research paper, 'Persistence diagrams of random matrices via Morse theory: universality and a new spectral diagnostic,' establishes a powerful new connection between algebraic topology and random matrix theory (RMT). The core finding is that the persistence diagram—a tool from topological data analysis (TDA) that tracks the 'birth' and 'death' of topological features like holes—of a quadratic form derived from a symmetric matrix M is analytically determined by M's eigenvalues. Specifically, applying Morse theory to the function f(x)=xᵀMx on a sphere shows each bar in the diagram corresponds to an eigenvalue spacing (λ_{k+1} - λ_k). This mathematical bridge allows the well-established universality laws of RMT (like the Gaussian Orthogonal Ensemble, or GOE) to transfer directly to the topological features, creating 'universal persistence diagrams' that act as fingerprints for different matrix ensembles.
As a practical consequence, the research introduces 'persistence entropy' (PE) as a novel spectral diagnostic tool. For GOE matrices of size n, the derived closed-form is PE = log(8n/π) - 1. This new metric is shown to outperform the standard level spacing ratio ⟨r⟩, a workhorse of RMT analysis. In tests discriminating GOE matrices from Gaussian Unitary Ensemble (GUE) matrices at n=100, persistence entropy achieved an Area Under the Curve (AUC) of 0.978 versus 0.952 for ⟨r⟩, with non-overlapping 95% confidence intervals. Furthermore, PE successfully detected global spectral perturbations in the Rosenzweig-Porter model that the standard ⟨r⟩ statistic was blind to, demonstrating it captures complementary information.
The work validates the numerical behavior of these topological statistics, showing the coefficient of variation decays as n^{-0.6}. By proving that different random matrix ensembles (GOE, GUE, Wishart) produce distinct, universal persistence diagrams, the paper provides a new 'topological fingerprinting' method. This fusion of TDA and RMT opens avenues for more robust analysis of complex systems where spectral properties are key, from quantum chaos to the eigenvalues of neural network weight matrices, offering a more sensitive tool for detecting deviations from randomness.
- Proves persistence diagram bars correspond to eigenvalue spacings (s_k = λ_{k+1} - λ_k), transferring RMT universality to TDA.
- Derives closed-form persistence entropy PE = log(8n/π) - 1 for GOE matrices and shows its variation decays as n^{-0.6}.
- New 'persistence entropy' metric achieves 0.978 AUC vs. 0.952 for standard tool in classifying matrix ensembles and detects hidden spectral perturbations.
Why It Matters
Provides a more sensitive, topology-based tool for analyzing complex systems in ML and physics, from neural networks to quantum data.