A Platform-Agnostic Multimodal Digital Human Modelling Framework: Neurophysiological Sensing in Game-Based Interaction
New framework uses OpenBCI Galea headset to capture EEG, EMG, EOG, PPG, and inertial data in game-based interactions.
A research team from Nottingham Trent University and Manchester Metropolitan University has published a novel framework for Digital Human Modeling (DHM) that decouples physiological data collection from AI inference. The core innovation is its platform-agnostic design, which uses the OpenBCI Galea headset as a unified sensing layer to capture five concurrent data streams: electroencephalography (EEG), electromyography (EMG), electrooculography (EOG), photoplethysmography (PPG), and inertial measurements. This multimodal data is collected during standardized, reproducible interactions within the open-source SuperTux game environment.
Rather than embedding specific AI models, the framework's primary output is structured, timestamped, and ethically prepared datasets. It models user interaction through computational task primitives and event markers, ensuring precise temporal alignment across all sensor modalities. This design explicitly supports future, ethics-approved AI research by providing 'AI-ready' data without pre-biasing the analysis pipeline. The authors verified the technical integrity through self-instrumentation, confirming data stream continuity and synchronization.
The proposed infrastructure directly addresses current limitations in DHM research, where approaches are often siloed to specific platforms or interpretive methods. By providing a scalable and reproducible foundation, the framework aims to accelerate inclusive HCI studies, particularly in accessibility-oriented interaction design and the development of adaptive AI systems that can respond to nuanced physiological states.
- Integrates OpenBCI Galea headset to capture 5 concurrent biosignal streams (EEG, EMG, EOG, PPG, inertial) for comprehensive physiological sensing.
- Uses the SuperTux game engine to create a reproducible, platform-agnostic interaction environment for consistent data collection.
- Explicitly separates data sensing from AI inference, outputting structured, timestamped datasets to enable diverse downstream AI research under ethical approval.
Why It Matters
Provides a standardized, ethical foundation for building adaptive AI systems that understand human cognitive and physiological states in real-time.