EEG-FM-Audit reveals tuned baselines beat advanced EEG foundation models

A new paper from Xianheng Wang, Yige Yang, and Damien Coyle introduces EEG-FM-Audit, a systematic evaluation and analysis pipeline designed to address critical gaps in large EEG foundation model (FM) research. The pipeline tackles three key limitations: opaque supervised baseline tuning, unverified complex learning paradigms, and lack of transparency in model decision-making. EEG-FM-Audit includes an ASHA-driven benchmarking protocol for fair comparisons, paradigm-level ablation studies, and a neurophysiological probing (NPP) framework to check if FMs use valid temporal, spatial, and spectral EEG properties.

Testing on four state-of-the-art EEG-FMs and five supervised models across three public datasets, the results reveal a surprising insight: properly tuned supervised baselines can match or outperform advanced FMs despite requiring significantly fewer parameters. Additionally, the effectiveness of FM learning paradigms heavily depends on dataset scale and architecture. The NPP analysis shows how FMs rely on specific physiological features, establishing a framework for more interpretable neural decoding. The 26-page paper is available on arXiv.