Contracting Neural Networks: Sharp LMI Conditions with Applications to Integral Control and Deep Learning

A team of researchers including Anand Gokhale, Anton V. Proskurnikov, Yu Kawano, and Francesco Bullo has published a significant paper titled 'Contracting Neural Networks: Sharp LMI Conditions with Applications to Integral Control and Deep Learning' on arXiv. The work provides rigorous mathematical conditions that guarantee neural networks will behave in stable, predictable ways—a property called contractivity. Specifically, they derive sharp Linear Matrix Inequality (LMI) conditions for both firing-rate and Hopfield recurrent neural networks, applicable to common activation functions and both continuous and discrete time settings. This establishes a direct link between network weights and guaranteed stability.

The research demonstrates two major applications of this theoretical framework. First, it enables the design of low-gain integral controllers for contracting firing-rate networks, allowing them to reliably track reference signals—a crucial capability for control systems. Second, and perhaps more impactful for AI development, the team provides an exact parameterization of weight matrices that ensure contraction. They applied this to Implicit Neural Networks (INNs), a class of models where the output is defined by solving an equilibrium equation. By constraining these networks to be contracting, they achieved competitive performance on standard image classification benchmarks while using fewer parameters, proving that stability does not have to come at the cost of expressivity or accuracy.