Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG

A research team including Hanning Guo, Farah Abdellatif, and Jürgen Dammers has unveiled Brain-OF, a groundbreaking AI model that represents the first unified foundation model capable of processing three major types of brain imaging data: functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and magnetoencephalography (MEG). Published on arXiv, this work addresses a critical limitation in neuroscience AI, where most existing models are confined to single modalities, preventing them from leveraging the complementary strengths and collective scale of multimodal brain data. Brain-OF's architecture is specifically designed to handle both unimodal and multimodal inputs within a single framework, promising a more holistic approach to decoding brain activity.

The technical innovation behind Brain-OF centers on reconciling the heterogeneous spatiotemporal resolutions of different brain signals. The team developed the Any-Resolution Neural Signal Sampler to project diverse signals into a shared semantic space. The model's backbone integrates DINT attention with a Sparse Mixture of Experts (MoE), where shared experts capture modality-invariant patterns and routed experts specialize in modality-specific semantics. For pretraining, the researchers introduced Masked Temporal-Frequency Modeling, a dual-domain objective that reconstructs brain signals in both time and frequency domains. Trained on a large-scale corpus of approximately 40 datasets, Brain-OF demonstrates superior performance across diverse downstream neuroscience tasks, setting a new benchmark for multimodal brain foundation models and paving the way for more integrated neurotechnology applications.