Engineering Resource-constrained Software Systems with DNN Components: a Concept-based Pruning Approach

A research team led by Federico Formica has published a novel method for compressing massive AI models by pruning them based on human-understandable concepts. Traditional pruning removes network parameters based purely on mathematical importance, often harming performance on specific tasks. This new 'concept-based pruning' technique instead analyzes which neurons activate for interpretable features—like 'edges,' 'colors,' or object classes—and uses that understanding to surgically remove parts of the network less critical to the system's actual requirements. This bridges a key gap in software engineering, where DNNs must be integrated into larger systems with strict memory, storage, and computational constraints.

The team validated their approach using the VGG-19 network and a dataset of over 26,000 RGB images. Their results show the method efficiently generates significantly smaller, yet still highly effective, pruned models. The pruned networks demonstrate greatly improved computational efficiency and performance, which is crucial for practical applications like embedded systems, IoT devices, or mobile apps. Furthermore, the framework provides engineers with configuration options to explore trade-offs, allowing them to tailor the pruning process to different practical scenarios, balancing size, speed, and accuracy based on the specific needs of the software system being built.