Multi-component ICA theory discovers learnability phases and competition

Independent Component Analysis (ICA) is a staple of unsupervised representation learning, but its high-dimensional theory has mostly focused on recovering a single component. In a new paper on arXiv (arXiv:2605.08552), researchers Eser Ilke Genc, Samet Demir, and Zafer Dogan tackle the multi-component case by developing an asymptotically exact mean-field theory for online ICA. They derive a closed system of ordinary differential equations (ODEs) for the overlap matrix between learned estimates and ground-truth components. This reveals a genuinely multi-component, initialization-driven phase structure: a decoupled regime where estimates align with distinct components and evolve independently, and a competition regime where overlapping initializations cause orthogonality-driven conflicts, slow reorientation, and delayed convergence.

Their steady-state analysis provides explicit learnability boundaries and competition conditions linking step size, data moments, and initialization. They find that larger higher-order moments and competition shrink the stable learning-rate window and increase convergence times. The theory also predicts a staircase phenomenon: the number of recoverable components changes discretely with the learning rate. Experiments on synthetic data and hyperspectral remote sensing data confirm the predicted trajectories and phase behavior. These results give practitioners a rigorous framework for tuning ICA algorithms in high-dimensional settings where multiple components must be separated simultaneously.