How Well Can We Decode Vowels from Auditory EEG -- A Rigorous Cross-Subject Benchmark with Honest Assessment

In a new arXiv preprint, Xiaoyang Li presents a rigorous cross-subject benchmark for decoding five vowels (a, e, i, o, u) from auditory EEG using the OpenNeuro ds006104 dataset (16 subjects, 61 channels, 256 Hz). Unlike prior studies that rely on within-subject evaluation or small cohorts, this work enforces strict leave-one-subject-out evaluation with training-only normalization and explicit anti-leakage checks. The study compares 14 pipelines spanning classical machine learning, deep learning, and Riemannian methods. The best full-feature model, XGBoost, achieves 24.5% accuracy (20% chance), while differential entropy features with LightGBM reach 25.5%. After multiple comparison correction, few pairwise model advantages remain significant, and classical methods prove competitive with deep learning in this low-signal regime.

Additional analyses—including ablation, pairwise vowel comparisons, within-subject cross-validation, ERP analysis, temporal generalization, and electrode importance—reveal that vowel information is real but weak, primarily carried by early transient auditory responses. The study provides a reproducible baseline for EEG-based phoneme decoding, releasing full code and evaluation scripts. These results highlight the challenges of decoding speech from EEG in real-world settings, especially across different subjects, and underscore that even the best models barely exceed chance levels. This work sets a sobering yet important benchmark for the field of brain-computer interfaces.