FM-Speech model cracks fine-grained speech understanding across 14 dimensions

Current speech large language models excel at basic tasks like speech recognition but fail to perceive fine-grained, multi-dimensional aspects of speech—such as micro-acoustic cues, acoustic scenes, and paralinguistic signals. This limits their ability to build truly perceptive and empathetic speech systems. The root causes: scarce high-quality expressive data, lack of fine-grained modeling for multiple attributes, and coarse-grained benchmarks. To address this, a team of researchers (Guojian Li et al.) proposed three contributions: a robust data curation pipeline that extracts high-quality spontaneous speech from audiovisual sources while handling complex acoustic environments, a new benchmark (FMSU-Bench) covering 14 speech attribute dimensions, and a model called FM-Speech.

FM-Speech is driven by decoupled attribute modeling and a progressive curriculum fine-tuning framework, which substantially elevates fine-grained, multi-dimensional acoustic perception. Evaluations on FMSU-Bench show that current speech LLMs still require significant improvement, while FM-Speech substantially outperforms existing open-source models, establishing a robust paradigm for real-world speech understanding. The work is published on arXiv (2605.12036) and provides both the benchmark and the model as a foundation for future research in multi-dimensional speech perception.