MED-VAE aligns brain activity across subjects without shared stimuli

A team of researchers (Angeliki Papathanasiou, Jascha Achterberg, Thomas E. Nichols, Rui Ponte Costa) has developed a new neural network architecture called MED-VAE (Multi-Encoder-Decoder Variational Autoencoder) that solves a longstanding problem in neuroscience: aligning brain activity across different individuals without requiring them to see the same images. Traditional alignment methods, such as hyperalignment, demand shared stimuli across subjects — a major limitation for real-world, naturalistic experiments where each person might view different content. MED-VAE overcomes this by using a pretrained artificial neural network (ANN) as a common representational scaffold, projecting each subject's unique neural responses into a shared latent space.

Using the Natural Scenes Dataset (fMRI recordings of visual cortex), the authors demonstrate that MED-VAE produces a common latent space with superior semantic organization compared to baseline methods. It achieves higher cross-subject alignment scores and maintains robust generalization to held-out stimuli — where traditional methods often fail. Moreover, the decoder branch can reconstruct original neural activity from the shared latent space without losing stimulus-driven signal. The practical payoff is direct cross-subject neural decoding: using one subject's brain data to predict what another subject is seeing, validated through image reconstruction tasks. The work, presented at the 9th Conference on Cognitive Computational Neuroscience (2026), opens doors for generalizable brain-computer interfaces and multi-subject studies with unconstrained naturalistic stimuli.