Toward using Speech to Sense Student Emotion in Remote Learning Environments

A research team led by Sargam Vyas, Bogdan Vlasenko, and Mathew Magimai.-Doss has published a paper exploring the use of speech AI to sense student emotions in asynchronous remote learning environments. The work addresses a critical gap: unlike in-person classrooms, remote learning often lacks emotional cues, which can hinder the learning experience. The researchers investigated whether speech recorded during specific 'self-control tasks'—designed to aid remote learning—exhibits perceptible variations along three key emotional dimensions: valence (positive/negative), arousal (calm/excited), and dominance (submissive/empowered).

To answer this, the team developed a novel dataset containing spontaneous monologue speech from students responding to these tasks. They then conducted both subjective listener evaluations and built automatic prediction models to analyze the data. Their investigations confirmed that speech from these tasks does carry detectable emotional signals. This breakthrough suggests that paralinguistic speech processing technology could be seamlessly integrated into remote learning platforms. The potential impact is significant, enabling systems to provide real-time, emotion-aware feedback and allowing instructional designers to adapt content dynamically based on student engagement and emotional state, ultimately aiming to make digital education more responsive and effective.