Hand-coded MLP weights memorize sequences linearly but lag behind trained models
Researchers hand-code MLP weights for sequence memorization and pose a challenge to close the efficiency gap.
Linda Linsefors and Lucius Bushnaq present a challenge on the AI Alignment Forum: hand-coding weights for small MLPs to memorize sequences. Their one-layer models store random input-output mappings with 90% accuracy, and the number of facts scales linearly with parameter count—just like trained models. But the hand-coded approach has a prefactor roughly 10× worse, meaning trained models pack more facts into each weight. A hybrid method (hand-coded input weights, learned output weights) also falls short.
The work stems from a broader goal to understand factual storage in LLMs. Prior evidence points to MLP layers as key-value memories, but a weight-level account is missing. By studying hand-coded solutions that mimic trained behavior, the authors aim to reverse-engineer how gradient descent encodes look-ups. The challenge invites others to close the efficiency gap, potentially revealing new principles of memorization in neural networks.
- Hand-coded one-layer MLPs memorize random length-2 sequences with 90% accuracy; capacity scales linearly with parameter count.
- The hand-coded scaling prefactor is about 10× worse than that of gradient-trained models on the same architecture.
- Partial hand-coding (input weights fixed, output weights learned) also underperforms, suggesting the gap is non-trivial.
Why It Matters
Clarifying how MLPs store facts helps interpret LLM memorization and could improve mechanistic interpretability.