Quantifying information stored in synaptic connections rather than in firing activities of neural networks

A new theoretical framework from researchers Xinhao Fan and Shreesh P. Mysore, published in Neural Computation, tackles a longstanding gap in neuroscience: quantifying information stored in synaptic connections rather than neural firing activity. While decades of work have focused on how neurons encode information through spike patterns, the actual memory storage in synaptic weights—the strengths of connections between neurons—lacked a formal information-theoretic measure. The authors address this using densely connected Hebbian networks performing autoassociative memory tasks, modeling stored data patterns as log-normal distributions.

Their framework derives analytical approximations for Shannon mutual information between data and singletons, pairs, and arbitrary n-tuples of synaptic connections. A key finding is synergistic interactions among synapses: the information encoded jointly by all synapses exceeds the sum of individual contributions. This supports distributed coding principles and formalizes heterogeneity in synaptic information encoding. The work has implications for both biological neural networks and artificial neural networks, offering a new lens for understanding memory storage and learning dynamics.