NeuroCommitSSM uses EEG-EMG-eyes to decide when to move assistive robots
A new framework predicts commit readiness, slashing false starts to 0.75 per 1000 rest windows.
NeuroCommitSSM, presented by Tipu Sultan and six co-authors and accepted at IROS 2026, tackles a critical gap in assistive robotics: not just what to do, but when to execute. The system synchronizes electroencephalography (EEG), electromyography (EMG), and eye-tracking (ET) to produce a continuous commit-readiness score c_t in [0,1]. A dwell-and-hysteresis filter converts this into discrete commit events, while a three-state finite-state supervisor (HOLD-ASSIST-COMMIT) gates execution only when the neural signal, real-time perception, inverse kinematics solvability, and collision-free planning all align.
In a study with 32 subjects performing five activities of daily living (ADL) aligned with the ICF, NeuroCommitSSM achieved 0.950 action-balanced accuracy and just 0.75 false commit events per 1,000 REST windows (FP/1k REST) under full-sensor conditions. Even in a challenging EEG-only dropout scenario, it maintained 0.785 balanced accuracy and 0.29 FP/1k REST, drastically outperforming a Temporal Convolutional Network baseline that produced 99.95 FP/1k REST. Hardware-in-the-loop validation on a Kinova Gen3 arm confirmed that feasibility-checked execution reduces decision instability without compromising task success. Code, datasets, and videos are available online.
- Combines EEG, EMG, and eye-tracking to predict a continuous commit-readiness score before robotic execution.
- Achieves 0.950 action-balanced accuracy with only 0.75 false commits per 1,000 REST windows in 32-subject ADL trials.
- Hardware validation on a Kinova Gen3 robot shows reduced false starts while maintaining task success, with strong performance even under sensor dropout.
Why It Matters
A smarter brain-robot interface that reduces false starts, making assistive manipulation safer for users with disabilities.