A "Lay" Introduction to "On the Complexity of Neural Computation in Superposition"

A viral blog post is making complex AI theory accessible by explaining the 2022 paper 'On the Complexity of Neural Computation in Superposition.' The author, who attempted to read and present the dense theoretical work in an hour, breaks down its core premise: why individual neurons in neural networks are so hard to interpret. The post tackles the frustrating phenomenon of 'polysemanticity,' where a single neuron activates for a confusing mix of unrelated concepts, like cats and plans for a robotic uprising. This shattered the early dream of finding simple, human-understandable 'cat neurons' or 'betray-all-humans neurons' inside AI models.

The post explains the leading theory behind this behavior: 'representational superposition.' In high-dimensional spaces, neural networks can efficiently pack exponentially more information by using vectors that are nearly, but not perfectly, orthogonal. This is loosely connected to the Johnson-Lindenstrauss lemma from mathematics. The 2022 paper formalized this concept, showing how networks trade off a small amount of interference between concepts for massive gains in representational capacity. This theoretical groundwork directly enabled a wave of subsequent research in mechanistic interpretability, leading to the techniques used today to extract and understand concepts within large language models (LLMs).