New Framework Reveals 66.3% Brain Decoding Gains Are Structural Leakage

A new paper by Xinyu Zhang and colleagues at arXiv addresses a critical flaw in non-invasive brain-to-language decoding: reported performance can be artificially inflated by non-neural sources like decoder priors, embedding-based metrics, and—most significantly—variable signal duration. The authors propose an auditing framework for stimulus-locked MEG-to-audio retrieval that decomposes apparent accuracy into three sources: structural shortcuts, window-level stimulus-locked evidence, and cross-window contextual aggregation. Their key diagnostic shows that signal-blind Gaussian noise achieves 66.3% Rank@1 (R@1) when allowed variable-length decoding windows, but collapses to near-chance performance once fixed-duration windows and stimulus-identity splits are enforced, isolating leakage from signal duration.

Under properly controlled fixed-window conditions, the framework recovers measurable MEG-audio discriminability. Further analysis reveals that 95.7% of Top-1 errors select the wrong sentence, localizing the bottleneck to sentence-level competition rather than word-level neural evidence. To address this, the authors introduce Group Context Bias (GCB), an inference-time additive logit bias that pools sentence-consistent evidence across windows. GCB shifts R@1 from 44% to 52% on the Gwilliams dataset and from 22% to 29% on MOUS under identical settings. Critically, GCB's effect collapses under random-grouping perturbations and vanishes when local MEG evidence is attenuated or near chance in EEG, validating it as a controlled source-attribution intervention. The paper concludes that brain-to-language performance must be source-attributed—not merely reported—to ensure claims reflect genuine neural decoding.