MindAlign decodes inner speech from fMRI with limited data

Decoding inner speech from non-invasive brain signals has been a long-standing challenge—no overt speech output, scarce training data, and high variability between individuals. Existing approaches typically require task-specific decoder fine-tuning, limiting scalability. A new paper on arXiv introduces MindAlign, a decoupled two-stage framework that sidesteps these issues. In the first stage, subject-specific neural-semantic alignment maps fMRI activity into a shared multimodal semantic space, extracting a latent 'sketch' of the internally generated sentence. The second stage integrates this sketch with visual context and feeds it to a frozen multimodal language model (e.g., a vision-language model) for open-ended text generation—no language model modification needed.

Experiments on fMRI data collected during silent image description show that MindAlign consistently outperforms both fMRI-only and random baselines. Crucially, the learned semantic-to-language projection can generalize across subjects when paired with subject-specific neural alignment, meaning a new participant only needs to train the lightweight alignment module rather than an entire decoder. These results demonstrate that neural signals modulate semantic content beyond image-driven priors, supporting a scalable, modular direction for brain-to-text decoding. While still a preprint under review, the work represents a significant step toward practical, non-invasive brain-computer interfaces for silent communication.