New AI Study Shows Smarter Way to Shrink Brain-Like Networks

Magnitude pruning of recurrent neural networks ignores how information flows between neurons. An earlier version proposed modeling RNN dependencies as a partially ordered set and retaining the meet-irreducible elements of its dependency lattice — but by Birkhoff's representation theorem, the meet-irreducible ideals of a finite poset are in bijection with its elements, so retaining them retains everything, and the critical-set criterion selects precisely the elements comparable to all others, of which a dense recurrent layer has none.

Condensing a thresholded dependency graph into strongly connected components does yield a well-defined dependency order, a graded order score, and an acyclic pruning rule, and for Cohen-Grossberg networks and LSTMs acyclic pruning makes local stability decompose over modules. Yet for a 128-unit LSTM pruned at matched sparsity on row-wise and chunked sequential MNIST (three seeds, equal training budgets), the original rules remove every recurrent weight or equal random column pruning; the order score is no better than degree, PageRank or norm; trained weights show little directional bias; and imposing acyclicity from initialisation costs 2.1 and 10.5 percentage points at 50% and 90% sparsity. In the networks studied, order structure must be imposed rather than discovered.