Paper argues Transformers are a 'Giant Hippocampus', not universal cortex

Jaeho Seol's 48-page paper, 'The Giant Hippocampus,' argues that modern AI's reliance on a single Transformer architecture for text, images, and speech is a fundamental structural mistake. Unlike the brain's cortex, which uses distinct layers and cell types for different cognitive tasks (e.g., dense Layer 4 for vision, thick Layers 5/6 for motion), AI has defaulted to a 'structural monoculture.' The paper traces how early successes like CNNs (local receptive fields, hierarchical depth) were abandoned due to the 'Hardware Lottery' that made Transformers computationally path-of-least-resistance. Even Mixture-of-Experts, often seen as diverse, merely partitions parameters among identical Transformer experts.

The paper uses functionalist analysis to show that Transformers are best understood as a functional analog of the hippocampal formation—a memory system—not a general-purpose cortex. This is likened to treating the entire brain as one giant Broca's area. The proposed alternative is a Heterogeneous Topological Network: a 'System of Systems' where each module has the inductive bias its specific computation requires (e.g., spatial encoding, temporal integration), and modules communicate through standardized interfaces. This design discipline emphasizes specifying modularity before training, using structural evidence as a design input rather than reverse-engineering from trained model behavior. The 23 figures support the argument with cytoarchitecture evidence from Brodmann to single-cell Patch-seq.