Convex Efficient Coding

Researchers William Dorrell, Peter Latham, and James Whittington have introduced a novel mathematical framework, 'Convex Efficient Coding,' that provides a unified, tractable approach to normative theories of neural representation. The work addresses a core question in neuroscience and AI: why neurons encode information the way they do. Normative theories model neural activity as the solution to an optimization problem, balancing information fidelity with efficiency, but existing models range from overly simple linear approximations to intractably complex deep networks. This new framework splits the difference by demonstrating that a large family of these optimization problems are mathematically convex when you optimize a 'representational similarity' matrix—built from the dot products of neural responses—rather than the activities directly.

This convexity breakthrough has significant implications. The identified family includes problems corresponding to certain linear and nonlinear neural networks, as well as established methods like modified semi-nonnegative matrix factorization (NMF) and nonnegative sparse coding. The authors leverage this tractability in three key ways: providing the first necessary and sufficient identifiability proof for a form of semi-NMF, showing a unique link between optimal representations and single-neuron tunings under certain conditions, and using the model's nonlinear capabilities to explain a fundamental difference between dense retinal codes and sparse cortical codes. Specifically, the framework explains why retinas optimally split signal coding into ON and OFF channels, while cortices do not. This work bridges theoretical neuroscience and machine learning, offering a new lens to analyze and understand both biological and artificial neural systems.