Hamadouche & Hussain's control-theory compression slashes DNN parameters 73-83% with minimal accuracy loss

A new paper from Anis Hamadouche and Amir Hussain introduces an empirical minimal-realisation approach for compressing deep neural networks by leveraging controllability and observability tests from control theory. Rather than pruning weights or quantising activations directly, the authors view a trained network as a depth-indexed nonlinear dynamical system. They construct data-driven reachability, observability, and balanced Gramians from hidden-state snapshots and output Jacobians, then estimate layer-wise ranks that serve as both diagnostic redundancy measures and actual compressed layer widths. This principled method directly targets hidden-state redundancy at the architectural level.

Experiments on MNIST and CIFAR-10 demonstrate strong results. On MNIST, a four-layer SiLU DNN was reduced from 1024 to 277 hidden states, achieving 72.95% state compression and 73.48% parameter compression while maintaining 95.45% accuracy (vs. 96.60% full model). On CIFAR-10, a larger SiLU DNN went from 4608 to 1339 states (70.94% state compression, 83.09% parameter compression), preserving accuracy from 54.45% to 54.44% and reducing CUDA inference latency by approximately 3×. The method outperformed projection-based reduction, unstructured/structured pruning, low-rank SVD, and dynamic INT8 quantisation baselines, offering a principled minimal-realisation criterion for designing compact neural architectures with little or no accuracy loss.