Beyond Object-Level Alignment: Do Brains and DNNs Preserve the Same Transformations?

A new paper by Yukiyasu Kamitani (arXiv, May 2026) tackles a fundamental question in neuroscience and AI: do brains and deep neural networks (DNNs) not only represent the same objects, but also transform stimuli in the same way? Traditional alignment metrics like representational similarity analysis (RSA) or encoding accuracy compare per-stimulus activity or dataset geometry. Inspired by category theory, Kamitani formalizes a different notion called "approximate naturality." The idea: if a transformation is applied to a stimulus (e.g., changing animacy), a model faithfully aligned with the brain should produce a consistent mapping regardless of whether the transformation is applied before or after converting neural activity to model activity. The Naturality Violation Score (NVS) quantifies deviations from this commutativity, normalized against a permutation null distribution.

As proof of concept, Kamitani applied NVS to fMRI responses from the GOD dataset (5 subjects) and three vision DNNs (likely variants of ResNet or VGG), using three World-Model proxy embeddings to define candidate transformation axes. The axis-resolved analysis uncovers a striking hierarchy: semantic axes like animacy align most strongly with higher visual cortex (e.g., HVC) and deeper DNN layers (NVS_animacy = 0.39, compared to 0.52 for the next-best axis and 1.0 for the null baseline). In contrast, low- and mid-level visual axes (e.g., orientation, color) align better with early visual cortex and shallower layers. Supporting analyses with 15 axes, dissociation tests against RSA/CKA, and anchor-ablation controls confirm that alignment is selective over candidate transformation families, not uniform. The NVS framework thus turns brain-DNN alignment into a test of jointly preserved structural transformations, opening the door to richer proxy spaces and controlled world-side experiments.