New framework lets audio AI compare emotions with 95% less data

Researchers at Carnegie Mellon University and the University of Texas at Dallas have introduced a novel framework for comparative reasoning in audio-language models, specifically targeting speech emotion recognition (SER). Published on arXiv (2606.24082), the work addresses a critical gap: while large audio-language models (LALMs) can reason about audio in isolation, they struggle with ordinal comparisons — determining which of two utterances exhibits higher arousal, valence, or dominance. The team, led by Abinay Reddy Naini, proposes a reasoning-guided framework that conditions an LALM on paired speech inputs and trains it using reasoning traces generated from both semantic audio descriptions and acoustic evidence derived from GeMAPS features. This dual-source approach enables interpretable comparative decisions. Additionally, the researchers employ direct preference optimization (DPO) to enforce stronger separation for emotional differences, improving the model's ability to rank emotions precisely.

The results are striking: the proposed framework improves preference prediction while requiring only 5% of the training data used by conventional ordinal SER systems. This data efficiency is a game-changer for emotion AI, where labeled datasets are scarce and expensive to produce. By combining semantic reasoning with acoustic evidence, the model not only makes better comparisons but also provides interpretable justifications for its decisions. The work opens the door to more human-like emotional understanding in voice assistants, mental health monitoring, and customer service AI, where accurately gauging emotional intensity is crucial. While the study focuses on SER, the methodology could extend to other comparative audio tasks like speaker verification or environmental sound ranking.