EEG Brain-Computer Interface Benchmark Reveals Extreme Subject Variability
216,714 evaluations show unique pipelines for almost every subject — here's how to handle it.
Robust EEG motor imagery decoding is notoriously limited by inter-individual variability. In this large-scale benchmark, researchers analyzed 216,714 raw evaluation rows from three public datasets (Cho2017, PhysionetMI, Zhou2016) using a standardized MOABB framework with two frequency bands and multiple preprocessing-feature-classification combinations. Covariance tangent-space projection (cov-tgsp) and Common Spatial Patterns (CSP) emerged as the strongest methodological families, but their relative performance depended heavily on the dataset. Crucially, aggregate rankings concealed massive subject-level heterogeneity: 42 distinct winning pipelines across 52 subjects in Cho2017, and 93 across 109 subjects in PhysionetMI. This means a one-size-fits-all approach fails for BCI.
To address this, the authors treated the benchmark as an empirical performance landscape and constructed compact portfolios of pipelines. They compared several strategies, with the simple Top-K Mean heuristic giving the best trade-off. A single global pipeline already retained 94.2% of the oracle (best possible per-subject accuracy) in Cho2017 and 81.8% in PhysionetMI. Remarkably, with K=12 pipelines, retention rose to 96.5% and 90.0% respectively. This demonstrates that subject-dependent heterogeneity can be exploited through small, personalized pipeline sets, making real-world BCI personalization much more practical.
- Analyzed 216,714 pipeline evaluations across 165 subjects from three datasets.
- Found 42 unique best pipelines for 52 subjects in Cho2017 — 93 for 109 in PhysionetMI.
- Top-K Mean portfolio with 12 pipelines recovers 96.5% of oracle performance in Cho2017.
Why It Matters
Enables efficient personalization of BCI systems using small pipeline portfolios instead of one-size-fits-all.