LLMs mimic human brain semantics, but fail on social and emotional dimensions
New MEG study reveals LLMs align with neural synchronization – but not fully for agency and affect.
Deep Dive
Researchers Hong et al. combined storytelling-listening MEG with interbrain encoding to compare how humans and five recent LLMs represent shared semantics across ten dimensions. Both human- and LLM-derived semantic spaces predicted speaker-listener neural synchronization. However, the largest divergences emerged for dimensions tied to agency, affect, and social experience. Larger LLMs showed closer alignment with human neural structure, but the overlap remained incomplete and dimension-dependent.
Key Points
- Five recent LLMs (including GPT-4, Claude 3.5) were tested alongside human raters across 10 semantic dimensions using MEG hyperscanning.
- LLM-derived semantic spaces predicted speaker-listener neural synchronization as well as human ratings did, but with systematic differences in representational geometry.
- LLMs diverged most on dimensions related to agency, emotion, and social experience, revealing a selective convergence with human shared semantics.
Why It Matters
Shows where AI understanding still diverges from human cognition, guiding safer, more aligned model development.