MD-SigLIP achieves SOTA brain-language decoding with margin-regularized alignment

A new arXiv paper introduces MD-SigLIP, a framework that tackles a core problem in brain-language decoding: whether decoded content truly reflects neural representations or is merely hallucinated by the language model. The authors—Jiaqi Wang, Huawen Hu, and Shu Zhang—address this by directly aligning brain activity embeddings with text embeddings in a shared semantic space, enabling retrieval-based decoding rather than generative reconstruction. This explicit alignment makes the correspondence between neural signals and language semantics measurable and interpretable, a key step beyond black-box language model generation.

Technically, MD-SigLIP builds on duplicate-aware sigmoid contrastive learning, which handles multiple positive semantic clusters. It adds a listwise margin-regularized term that enforces structured ranking between positive clusters and negative samples, capturing the manifold organization of language embeddings reflected in neural signals. The combination allows the model to model multi-positive semantic structure and margin-based ordering simultaneously. Experiments show state-of-the-art retrieval performance in both full-vocabulary and subset settings, suggesting the framework reliably maps brain activity to specific linguistic concepts.