AI models converge like biology: Same tasks forge similar neural structures
Hungarian matching reveals hidden neurons align across independently trained networks—even before accuracy stabilizes.
In evolutionary biology, unrelated species often develop analogous traits under similar environmental pressures. A new paper from arXiv (2608.05985) by Patrick Krauss and colleagues asks whether the same happens in artificial neural networks: do randomly initialized networks trained on identical tasks converge to similar internal weight structures? The challenge is that hidden neurons can be permuted without changing the network's function, making naive matrix comparisons meaningless.
To solve this, the team introduced a matching-based framework that coarsely aligns hidden neurons using permutation-invariant features, then refines the alignment via iterative Hungarian matching. After alignment, they compared networks using structural distance metrics designed to emphasize task-relevant weight patterns. Testing ensembles of small multilayer perceptrons on MNIST, Fashion-MNIST, and KMNIST, they found that networks trained on the same task remained significantly closer to each other than to networks trained on different tasks, even across different datasets.
The most striking finding came from examining the earliest training phase. Classification accuracy rose rapidly before matched structural distances showed task-specific separation, and before global weight distributions visibly changed. Yet individual weight entries had already begun to drift in a coordinated, distributed fashion. This suggests early learning operates through subtle, widespread adjustments that dramatically affect function while leaving coarse network morphology nearly intact. The authors describe this as a first glimpse of a richer dynamical process—an 'early morphogenesis' that could reshape our understanding of how neural networks develop task-specific representations and whether algorithmic evolution mimics biological convergence.
- Introduces a matching-based framework (permutation-invariant features + Hungarian matching) to compare MLP weight structures across random initializations.
- Networks trained on the same task (MNIST, Fashion-MNIST, KMNIST) are structurally closer to each other than to networks trained on different tasks.
- Early training shows rapid accuracy gains before structural separation appears, revealing coordinated weight drift that precedes macroscopic morphological changes.
Why It Matters
This study suggests AI training follows deterministic attractors, potentially enabling better weight initialization, model merging, and interpretability across architectures.