Brain-Grasp: Graph-based Saliency Priors for Improved fMRI-based Visual Brain Decoding

A research team led by Mohammad Moradi and Morteza Moradi has introduced Brain-Grasp, a new framework that significantly advances the field of visual brain decoding. The system addresses a core limitation in existing fMRI-to-image technology: the frequent loss of object-level structure and semantic fidelity. Instead of generating blurry or conceptually inconsistent pictures from brain scans, Brain-Grasp employs a novel graph-based saliency prior. This technique translates structural cues from fMRI signals into precise spatial masks that outline where important objects are located in a scene.

These saliency masks are then combined with semantic information extracted from neural embeddings to condition a single, frozen diffusion model. This streamlined, one-stage architecture is a key innovation, as it replaces complex multi-model pipelines with a more efficient and effective design. The result is a system that guides image regeneration with a stronger grasp of scene composition, ensuring objects are placed correctly and maintain their real-world relationships.

Experiments demonstrate that Brain-Grasp delivers substantial improvements in both conceptual alignment and structural similarity to the original visual stimuli viewed by a subject. By grounding the decoding process in interpretable saliency graphs, the research also opens a new direction for creating efficient and structurally sound brain-computer interfaces. The work, detailed in the arXiv preprint 2604.10617, represents a meaningful step toward more accurate and reliable reconstruction of visual experiences directly from neural activity.