A Hierarchical Importance-Guided Multi-objective Evolutionary Framework for Deep Neural Network Pruning

Researchers Zak Khan and Azam Asilian Bidgoli have introduced a novel AI model compression technique called a Hierarchical Importance-Guided Multi-objective Evolutionary Framework. The core challenge it addresses is pruning over-parameterized deep neural networks—a massive, non-convex optimization problem where traditional methods often get stuck or are too computationally expensive. This new framework reformulates pruning as a tractable, large-scale multi-objective search, aiming to find the optimal trade-off between model compactness (size) and accuracy.

The method works in two distinct phases. First, a continuous evolutionary search performs a broad, coarse exploration across all network weights to identify promising regions for pruning and drastically shrink the search space. Second, a fine-grained binary evolutionary optimization, constrained only to the surviving weights from phase one, uses importance-aware sampling to refine the search. This hierarchical design combines global exploration with localized exploitation to efficiently discover a diverse set of optimal, pruned networks.

Empirical validation on standard image classification datasets CIFAR-10 and CIFAR-100 using ResNet-56 and ResNet-110 models demonstrated significant gains. The framework achieved parameter reductions of up to 51.9% and 38.9% respectively, with almost no drop in accuracy compared to the original dense models. This performance surpasses existing state-of-the-art evolutionary approaches to DNN pruning. The authors position the work as a general, scalable paradigm for solving very-large-scale multi-objective problems, potentially extendable to other domains beyond AI model compression.