New SOC Layer Boosts Speech Emotion Recognition by Preserving Feature Geometry
A novel second-order correlation layer recovers lost discriminative information in self-supervised speech models.
Self-supervised learning (SSL) produces powerful representations for speech emotion recognition (SER), but aggregating these high-dimensional features into a holistic descriptor remains a bottleneck. Traditional first-order pooling methods (e.g., mean or max) implicitly assume features are independent, ignoring the latent Riemannian geometry of SSL embeddings. This discards higher-order relationships that are crucial for capturing nuanced emotional cues. A new paper from researchers Shuanglin Li, Ruxiao Qian, and Siyang Song tackles this by introducing a Second-Order Correlation (SOC) layer.
The SOC layer explicitly models feature correlations as covariance descriptors, treating them as points on a Riemannian manifold. By applying Log-Euclidean mapping (LEM), the method preserves the geometric integrity of these descriptors while mapping them to a Euclidean tangent space suitable for linear discriminative learning. This approach captures synergistic co-occurrence patterns between features, forming discriminative signatures for emotion recognition. Experiments on the ESD and RAVDESS datasets demonstrate that SOC significantly outperforms first-order pooling, recovering lost discriminative information and effectively aggregating high-dimensional SSL features for more robust SER.
- SOC layer models feature correlations as covariance descriptors on a Riemannian manifold, capturing co-occurrence patterns.
- Log-Euclidean mapping preserves geometric structure while enabling linear discriminative learning.
- Outperforms first-order pooling on ESD and RAVDESS speech emotion recognition benchmarks.
Why It Matters
Enables more accurate emotion detection from speech, improving mental health monitoring, virtual assistants, and human-computer interaction.